mcesptool/PROJECT_SETUP.md

# 🚀 FastMCP ESPTool Server - Project Setup & Initialization

## Quick Start

```bash
# 1. Clone and initialize project
git clone <repository-url> mcp-esptool-server
cd mcp-esptool-server

# 2. Set up Python environment with uv
uv venv
uv pip install -e ".[dev]"

# 3. Start development server
uv run mcp-esptool-server

# 4. Add to Claude Desktop
claude mcp add esptool "uv run mcp-esptool-server"
```

## 📁 Project Structure

```
mcp-esptool-server/
├── src/
│   └── mcp_esptool_server/
│       ├── __init__.py
│       ├── server.py                    # Main FastMCP server
│       ├── config.py                    # Configuration management
│       ├── middleware/                  # Middleware system
│       │   ├── __init__.py
│       │   ├── logger_interceptor.py    # Base middleware classes
│       │   ├── esptool_middleware.py    # ESPTool-specific middleware
│       │   └── idf_middleware.py        # ESP-IDF middleware
│       └── components/                  # Tool components
│           ├── __init__.py
│           ├── chip_control.py          # ESP chip operations
│           ├── flash_manager.py         # Flash memory management
│           ├── partition_manager.py     # Partition table tools
│           ├── security_manager.py      # Security & eFuse tools
│           ├── firmware_builder.py      # Binary processing
│           ├── ota_manager.py           # OTA update tools
│           ├── production_tools.py      # Factory programming
│           ├── diagnostics.py           # Debug & analysis
│           └── idf_host_apps.py         # ESP-IDF host applications
├── tests/
│   ├── unit/                           # Unit tests
│   ├── integration/                    # Integration tests
│   ├── end_to_end/                    # Full workflow tests
│   └── fixtures/                      # Test data
├── examples/                          # Usage examples
├── docs/                             # Additional documentation
├── scripts/                          # Development scripts
├── docker/                           # Docker configuration
├── .env.example                      # Environment template
├── .gitignore
├── Dockerfile
├── docker-compose.yml
├── Makefile
├── pyproject.toml
└── README.md
```

## 🔧 Configuration Setup

### Environment Variables

```bash
# .env file configuration
# Copy from .env.example and customize

# Core paths
ESPTOOL_PATH=esptool                     # esptool binary path
ESP_IDF_PATH=/opt/esp-idf                # ESP-IDF installation
MCP_PROJECT_ROOTS=/workspace/projects    # Project directories

# Communication settings
ESP_DEFAULT_BAUD_RATE=460800            # Default baud rate
ESP_CONNECTION_TIMEOUT=30               # Connection timeout (seconds)
ESP_ENABLE_STUB_FLASHER=true           # Enable stub flasher

# Middleware options
MCP_ENABLE_PROGRESS=true                # Enable progress tracking
MCP_ENABLE_ELICITATION=true            # Enable user interaction
MCP_LOG_LEVEL=INFO                     # Logging level

# Development settings
DEV_ENABLE_HOT_RELOAD=true             # Hot reload in development
DEV_MOCK_HARDWARE=false                # Mock hardware for testing
DEV_ENABLE_TRACING=false               # Enable detailed tracing

# Production settings
PROD_ENABLE_SECURITY_AUDIT=true        # Security auditing
PROD_REQUIRE_CONFIRMATIONS=true        # User confirmations
PROD_MAX_CONCURRENT_OPERATIONS=5       # Concurrent operation limit
```

### Python Configuration

```toml
# pyproject.toml
[project]
name = "mcp-esptool-server"
version = "2025.09.28.1"
description = "FastMCP server for ESP32/ESP8266 development with esptool integration"
readme = "README.md"
requires-python = ">=3.10"
license = { text = "MIT" }
authors = [
    { name = "Your Name", email = "your.email@example.com" }
]

keywords = [
    "mcp", "model-context-protocol", "esp32", "esp8266",
    "esptool", "esp-idf", "embedded", "iot", "fastmcp"
]

classifiers = [
    "Development Status :: 4 - Beta",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Topic :: Software Development :: Embedded Systems",
    "Topic :: Scientific/Engineering :: Artificial Intelligence",
]

dependencies = [
    "fastmcp>=2.12.4",              # FastMCP framework
    "esptool>=5.0.0",               # ESPTool Python API
    "pyserial>=3.5",                # Serial communication
    "pyserial-asyncio>=0.6",        # Async serial support
    "thefuzz[speedup]>=0.22.1",     # Fuzzy string matching
    "pydantic>=2.0.0",              # Data validation
    "click>=8.0.0",                 # CLI framework
    "rich>=13.0.0",                 # Rich console output
]

[project.optional-dependencies]
dev = [
    "pytest>=8.4.2",
    "pytest-asyncio>=0.21.0",
    "pytest-cov>=7.0.0",
    "pytest-mock>=3.12.0",
    "ruff>=0.13.2",
    "mypy>=1.5.0",
    "black>=23.0.0",
    "pre-commit>=3.0.0",
    "watchdog>=3.0.0",            # Hot reload support
]

idf = [
    "esp-idf-tools>=2.0.0",       # ESP-IDF integration
    "kconfiglib>=14.1.0",         # Kconfig parsing
]

testing = [
    "pytest-xdist>=3.0.0",       # Parallel testing
    "pytest-benchmark>=4.0.0",    # Performance testing
    "factory-boy>=3.3.0",        # Test data factories
]

docs = [
    "mkdocs>=1.5.0",
    "mkdocs-material>=9.0.0",
    "mkdocs-mermaid2-plugin>=1.0.0",
]

[project.scripts]
mcp-esptool-server = "mcp_esptool_server.server:main"
esptool-mcp = "mcp_esptool_server.cli:cli"

[project.urls]
Homepage = "https://github.com/yourusername/mcp-esptool-server"
Repository = "https://github.com/yourusername/mcp-esptool-server"
Issues = "https://github.com/yourusername/mcp-esptool-server/issues"
Documentation = "https://yourusername.github.io/mcp-esptool-server"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
packages = ["src/mcp_esptool_server"]

[tool.ruff]
line-length = 100
target-version = "py310"
src = ["src", "tests"]

[tool.ruff.lint]
select = ["E", "F", "W", "B", "I", "N", "UP", "ANN", "S", "C4", "DTZ", "T20"]
ignore = ["E501", "ANN101", "ANN102", "S101"]

[tool.ruff.lint.per-file-ignores]
"tests/*" = ["S101", "ANN"]

[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
strict_optional = true

[[tool.mypy.overrides]]
module = "esptool.*"
ignore_missing_imports = true

[tool.pytest.ini_options]
testpaths = ["tests"]
asyncio_mode = "auto"
addopts = [
    "--cov=src/mcp_esptool_server",
    "--cov-report=html",
    "--cov-report=term-missing",
    "--cov-fail-under=85"
]

[tool.coverage.run]
source = ["src"]
omit = ["tests/*", "*/test_*"]

[tool.coverage.report]
exclude_lines = [
    "pragma: no cover",
    "def __repr__",
    "raise AssertionError",
    "raise NotImplementedError",
]
```

## 🐳 Docker Development Environment

### docker-compose.yml

```yaml
# Development environment with hot reload
services:
  mcp-esptool-server:
    build:
      context: .
      dockerfile: Dockerfile
      target: ${MODE:-development}
    container_name: mcp-esptool-dev
    environment:
      - LOG_LEVEL=${LOG_LEVEL:-DEBUG}
      - MCP_PROJECT_ROOTS=/workspace/projects
      - ESPTOOL_PATH=/usr/local/bin/esptool
      - ESP_IDF_PATH=/opt/esp-idf
      - DEV_ENABLE_HOT_RELOAD=true
    volumes:
      # Development volumes for hot reload
      - ./src:/app/src:ro
      - ./tests:/app/tests:ro
      - ./examples:/app/examples:ro
      # Project workspace
      - esp-projects:/workspace/projects
      # ESP-IDF and tools
      - esp-idf:/opt/esp-idf
      - esp-tools:/opt/esp-tools
      # Device access (Linux)
      - /dev:/dev:ro
    ports:
      - "${SERVER_PORT:-8080}:8080"
    networks:
      - default
      - caddy
    labels:
      # Caddy reverse proxy
      caddy: ${CADDY_DOMAIN:-esp-tools.local}
      caddy.reverse_proxy: "{{upstreams 8080}}"
    restart: unless-stopped
    develop:
      watch:
        - action: sync
          path: ./src
          target: /app/src
        - action: rebuild
          path: pyproject.toml

  # ESP-IDF development environment
  esp-idf-dev:
    image: espressif/idf:latest
    container_name: esp-idf-tools
    volumes:
      - esp-idf:/opt/esp-idf
      - esp-tools:/opt/esp-tools
      - esp-projects:/workspace/projects
    command: tail -f /dev/null
    networks:
      - default

volumes:
  esp-projects:
    name: ${COMPOSE_PROJECT_NAME:-mcp-esptool}-projects
  esp-idf:
    name: ${COMPOSE_PROJECT_NAME:-mcp-esptool}-idf
  esp-tools:
    name: ${COMPOSE_PROJECT_NAME:-mcp-esptool}-tools

networks:
  default:
    name: ${COMPOSE_PROJECT_NAME:-mcp-esptool}
  caddy:
    external: true
```

### Multi-stage Dockerfile

```dockerfile
# Dockerfile
# Base image with Python and system dependencies
FROM python:3.11-slim-bookworm AS base

# Install system dependencies
RUN apt-get update && apt-get install -y \
    git \
    curl \
    wget \
    build-essential \
    cmake \
    ninja-build \
    libusb-1.0-0-dev \
    && rm -rf /var/lib/apt/lists/*

# Install uv for fast Python package management
RUN pip install uv

# Create app user
RUN useradd --create-home --shell /bin/bash app
USER app
WORKDIR /app

# Development stage
FROM base AS development

# Copy project files
COPY --chown=app:app pyproject.toml ./
COPY --chown=app:app src/ ./src/

# Install development dependencies
RUN uv venv && \
    uv pip install -e ".[dev,idf,testing]"

# Install esptool and ESP-IDF
RUN uv pip install esptool
ENV PATH="/home/app/.local/bin:$PATH"

# Configure ESP-IDF (development version)
RUN git clone --depth 1 --branch v5.1 https://github.com/espressif/esp-idf.git /opt/esp-idf
RUN cd /opt/esp-idf && ./install.sh esp32

# Set up environment
ENV ESP_IDF_PATH=/opt/esp-idf
ENV PATH="$ESP_IDF_PATH/tools:$PATH"

EXPOSE 8080
CMD ["uv", "run", "mcp-esptool-server"]

# Production stage
FROM base AS production

# Copy only necessary files
COPY --chown=app:app pyproject.toml ./
COPY --chown=app:app src/ ./src/

# Install production dependencies only
RUN uv venv && \
    uv pip install -e ".[idf]" --no-dev

# Install production tools
RUN uv pip install esptool
ENV PATH="/home/app/.local/bin:$PATH"

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
    CMD curl -f http://localhost:8080/health || exit 1

EXPOSE 8080
CMD ["uv", "run", "mcp-esptool-server", "--production"]
```

## 🛠️ Development Tools

### Makefile

```makefile
# Makefile for development tasks
.PHONY: help install dev test lint format clean docker-build docker-up docker-down

# Default target
help:
	@echo "Available targets:"
	@echo "  install     - Install dependencies"
	@echo "  dev         - Start development server"
	@echo "  test        - Run tests"
	@echo "  lint        - Run linting"
	@echo "  format      - Format code"
	@echo "  clean       - Clean build artifacts"
	@echo "  docker-up   - Start Docker development environment"
	@echo "  docker-down - Stop Docker environment"

# Installation
install:
	uv venv
	uv pip install -e ".[dev,idf,testing]"

# Development server
dev:
	uv run mcp-esptool-server --debug

# Testing
test:
	uv run pytest tests/ -v

test-cov:
	uv run pytest tests/ --cov=src --cov-report=html --cov-report=term

test-integration:
	uv run pytest tests/integration/ -v

# Code quality
lint:
	uv run ruff check src/ tests/
	uv run mypy src/

format:
	uv run ruff format src/ tests/
	uv run ruff check --fix src/ tests/

# Cleanup
clean:
	find . -type d -name "__pycache__" -exec rm -rf {} +
	find . -type f -name "*.pyc" -delete
	rm -rf dist/ build/ *.egg-info/
	rm -rf htmlcov/ .coverage
	rm -rf .pytest_cache/

# Docker operations
docker-build:
	docker compose build

docker-up:
	docker compose up -d

docker-down:
	docker compose down

docker-logs:
	docker compose logs -f mcp-esptool-server

# ESP-IDF specific
esp-idf-setup:
	docker compose exec esp-idf-dev /opt/esp-idf/install.sh

# Add to Claude Desktop
claude-add:
	claude mcp add esptool "uv run mcp-esptool-server"

# Remove from Claude Desktop
claude-remove:
	claude mcp remove esptool
```

## 🔧 Development Scripts

### scripts/setup.py

```python
#!/usr/bin/env python3
"""Setup script for MCP ESPTool Server development environment."""

import os
import subprocess
import sys
from pathlib import Path

def run_command(cmd: str, check: bool = True) -> subprocess.CompletedProcess:
    """Run command and handle errors."""
    print(f"Running: {cmd}")
    result = subprocess.run(cmd, shell=True, capture_output=True, text=True)

    if check and result.returncode != 0:
        print(f"Error running command: {cmd}")
        print(f"Stdout: {result.stdout}")
        print(f"Stderr: {result.stderr}")
        sys.exit(1)

    return result

def setup_python_environment():
    """Set up Python environment with uv."""
    print("🐍 Setting up Python environment...")

    # Check if uv is installed
    result = run_command("uv --version", check=False)
    if result.returncode != 0:
        print("Installing uv...")
        run_command("pip install uv")

    # Create virtual environment and install dependencies
    run_command("uv venv")
    run_command("uv pip install -e '.[dev,idf,testing]'")

    print("✅ Python environment ready!")

def setup_esptool():
    """Set up esptool."""
    print("🔧 Setting up esptool...")
    run_command("uv pip install esptool")

    # Verify installation
    result = run_command("uv run esptool version", check=False)
    if result.returncode == 0:
        print("✅ ESPTool ready!")
    else:
        print("⚠️ ESPTool installation may have issues")

def setup_esp_idf():
    """Set up ESP-IDF (optional)."""
    print("🏗️ Checking ESP-IDF installation...")

    esp_idf_path = os.environ.get('ESP_IDF_PATH')
    if esp_idf_path and Path(esp_idf_path).exists():
        print(f"✅ ESP-IDF found at {esp_idf_path}")
    else:
        print("⚠️ ESP-IDF not found. Install manually or use Docker environment.")
        print("   Docker: make docker-up")
        print("   Manual: https://docs.espressif.com/projects/esp-idf/en/latest/get-started/")

def setup_git_hooks():
    """Set up git pre-commit hooks."""
    print("🪝 Setting up git hooks...")

    if Path(".git").exists():
        run_command("uv run pre-commit install")
        print("✅ Git hooks installed!")
    else:
        print("⚠️ Not a git repository, skipping hooks")

def create_env_file():
    """Create .env file from template."""
    print("📝 Creating environment file...")

    env_file = Path(".env")
    env_example = Path(".env.example")

    if not env_file.exists() and env_example.exists():
        env_file.write_text(env_example.read_text())
        print("✅ Created .env file from template")
        print("   Please review and customize .env file")
    else:
        print("ℹ️ .env file already exists or no template found")

def verify_installation():
    """Verify the installation."""
    print("🔍 Verifying installation...")

    # Test import
    result = run_command("uv run python -c 'import mcp_esptool_server; print(\"Import successful\")'", check=False)
    if result.returncode == 0:
        print("✅ Package import successful!")
    else:
        print("❌ Package import failed!")
        return False

    # Test server startup (dry run)
    result = run_command("uv run mcp-esptool-server --help", check=False)
    if result.returncode == 0:
        print("✅ Server startup test successful!")
    else:
        print("❌ Server startup test failed!")
        return False

    return True

def main():
    """Main setup function."""
    print("🚀 Setting up MCP ESPTool Server development environment...")

    # Change to project directory
    project_root = Path(__file__).parent.parent
    os.chdir(project_root)

    try:
        setup_python_environment()
        setup_esptool()
        setup_esp_idf()
        setup_git_hooks()
        create_env_file()

        if verify_installation():
            print("\n🎉 Setup completed successfully!")
            print("\nNext steps:")
            print("1. Review and customize .env file")
            print("2. Start development server: make dev")
            print("3. Add to Claude Desktop: make claude-add")
            print("4. Or use Docker: make docker-up")
        else:
            print("\n❌ Setup completed with errors. Check output above.")
            sys.exit(1)

    except KeyboardInterrupt:
        print("\n⚠️ Setup interrupted by user")
        sys.exit(1)
    except Exception as e:
        print(f"\n❌ Setup failed: {e}")
        sys.exit(1)

if __name__ == "__main__":
    main()
```

## 🚀 Getting Started Guide

### 1. Initial Setup

```bash
# Clone repository
git clone <repository-url> mcp-esptool-server
cd mcp-esptool-server

# Run automated setup
python scripts/setup.py

# Or manual setup
make install
```

### 2. Configuration

```bash
# Copy and customize environment
cp .env.example .env
# Edit .env with your settings

# Set up ESP-IDF (optional)
# Option A: Use Docker
make docker-up

# Option B: Install locally
# Follow: https://docs.espressif.com/projects/esp-idf/en/latest/get-started/
```

### 3. Development

```bash
# Start development server
make dev

# Run tests
make test

# Code formatting
make format

# Add to Claude Desktop
make claude-add
```

### 4. Verification

```bash
# Test basic functionality
uv run python -c "
import asyncio
from mcp_esptool_server.server import main
print('✅ Server imports successfully')
"

# Test with Claude
claude mcp list
# Should show 'esptool' server
```

This setup provides a complete development environment with modern Python tooling, Docker support, and comprehensive testing infrastructure.

`★ Insight ─────────────────────────────────────`
**Modern Python Project Structure**: This setup follows current best practices with uv for dependency management, ruff for linting/formatting, and comprehensive testing infrastructure. The multi-stage Docker setup enables both development and production deployments.

**Development Experience Focus**: The Makefile and scripts prioritize developer productivity with hot reload, automated testing, and easy Claude Desktop integration. The setup script automates the entire development environment creation.

**Production Readiness**: The configuration separates development and production concerns while maintaining compatibility with your preferred infrastructure patterns (Caddy, Docker Compose, environment variables).
`─────────────────────────────────────────────────`

<function_calls>
<invoke name="TodoWrite">
<parameter name="todos">[{"content": "Create project initialization and setup documentation", "status": "completed", "activeForm": "Creating project initialization and setup documentation"}, {"content": "Design concrete implementation examples", "status": "in_progress", "activeForm": "Designing concrete implementation examples"}, {"content": "Create production deployment guide", "status": "pending", "activeForm": "Creating production deployment guide"}]