MCP/mcesptool
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
mcesptool/MIDDLEWARE_DESIGN_PATTERNS.md
Ryan Malloy 64c1505a00 Add QEMU ESP32 emulation support 
Integrate Espressif's QEMU fork for virtual ESP device management:

- QemuManager component with 5 MCP tools (start/stop/list/status/flash)
- Config auto-detects QEMU binaries from ~/.espressif/tools/
- Supports esp32, esp32s2, esp32s3, esp32c3 chip emulation
- Virtual serial over TCP (socket://localhost:PORT) transparent to esptool
- Scan integration: QEMU instances appear in esp_scan_ports results
- Blank flash images initialized to 0xFF (erased NOR flash state)
- 38 unit tests covering lifecycle, port allocation, flash writes
2026-01-28 15:35:22 -07:00

630 lines
21 KiB
Markdown
Raw Blame History

# 🎨 Reusable MCP Middleware Design Patterns
## Overview
This document establishes reusable design patterns for creating MCP middleware that can integrate any CLI tool with Model Context Protocol servers. These patterns provide tested, scalable solutions for common integration challenges.
## 🏗️ Core Design Patterns
### 1. **Adapter Pattern - Tool Interface Adaptation**
Adapt different CLI tool interfaces to a common MCP integration standard.
```python
# patterns/adapter.py
from abc import ABC, abstractmethod
from typing import Any, Dict, List, Optional, Callable
from fastmcp import Context
class ToolAdapter(ABC):
"""Abstract adapter for CLI tool integration"""
def __init__(self, context: Context, operation_id: str):
self.context = context
self.operation_id = operation_id
self.capabilities = self._detect_mcp_capabilities()
@abstractmethod
def get_logging_interface(self) -> Dict[str, Callable]:
"""Return mapping of tool's logging methods to middleware handlers"""
pass
@abstractmethod
def get_progress_interface(self) -> Optional[Callable]:
"""Return tool's progress reporting mechanism"""
pass
@abstractmethod
def get_interaction_points(self) -> List[str]:
"""Return list of operations that require user interaction"""
pass
@abstractmethod
def install_hooks(self) -> None:
"""Install middleware hooks into tool"""
pass
@abstractmethod
def remove_hooks(self) -> None:
"""Remove middleware hooks from tool"""
pass
def _detect_mcp_capabilities(self) -> Dict[str, bool]:
"""Detect available MCP client capabilities"""
return {
'progress': hasattr(self.context, 'progress'),
'elicitation': hasattr(self.context, 'request_user_input'),
'logging': hasattr(self.context, 'log'),
'sampling': hasattr(self.context, 'sample')
}
```
### 2. **Strategy Pattern - Multiple Integration Strategies**
Support different integration approaches based on tool capabilities.
```python
# patterns/strategy.py
from enum import Enum
from typing import Protocol, runtime_checkable
class IntegrationStrategy(Enum):
LOGGER_REPLACEMENT = "logger_replacement"
OUTPUT_CAPTURE = "output_capture"
SUBPROCESS_WRAPPER = "subprocess_wrapper"
API_HOOKS = "api_hooks"
@runtime_checkable
class IntegrationHandler(Protocol):
"""Protocol for integration strategy handlers"""
async def integrate(self, tool_instance: Any, middleware: ToolAdapter) -> None:
"""Integrate middleware with tool using this strategy"""
...
async def cleanup(self, tool_instance: Any) -> None:
"""Clean up integration"""
...
class LoggerReplacementStrategy:
"""Replace tool's logger with MCP-aware version"""
async def integrate(self, tool_instance: Any, middleware: ToolAdapter) -> None:
"""Replace logger with middleware version"""
logging_interface = middleware.get_logging_interface()
# Store original logger
if hasattr(tool_instance, '_original_logger'):
middleware._original_logger = tool_instance._original_logger
# Install MCP logger
mcp_logger = self._create_mcp_logger(middleware, logging_interface)
self._install_logger(tool_instance, mcp_logger)
async def cleanup(self, tool_instance: Any) -> None:
"""Restore original logger"""
if hasattr(tool_instance, '_original_logger'):
self._install_logger(tool_instance, tool_instance._original_logger)
def _create_mcp_logger(self, middleware: ToolAdapter, interface: Dict) -> Any:
"""Create MCP-integrated logger for tool"""
# Implementation specific to tool's logger interface
pass
def _install_logger(self, tool_instance: Any, logger: Any) -> None:
"""Install logger in tool"""
# Implementation specific to tool's logger mechanism
pass
class OutputCaptureStrategy:
"""Capture tool's stdout/stderr and translate to MCP"""
async def integrate(self, tool_instance: Any, middleware: ToolAdapter) -> None:
"""Set up output capture"""
import sys
from io import StringIO
# Capture stdout/stderr
middleware._original_stdout = sys.stdout
middleware._original_stderr = sys.stderr
# Install capturing streams
middleware._stdout_capture = MCPOutputStream(middleware.context, 'stdout')
middleware._stderr_capture = MCPOutputStream(middleware.context, 'stderr')
sys.stdout = middleware._stdout_capture
sys.stderr = middleware._stderr_capture
async def cleanup(self, tool_instance: Any) -> None:
"""Restore original streams"""
import sys
if hasattr(middleware, '_original_stdout'):
sys.stdout = middleware._original_stdout
sys.stderr = middleware._original_stderr
class SubprocessWrapperStrategy:
"""Wrap tool as subprocess and capture communication"""
async def integrate(self, tool_instance: Any, middleware: ToolAdapter) -> None:
"""Set up subprocess wrapper"""
# Implementation for subprocess-based tools
pass
async def cleanup(self, tool_instance: Any) -> None:
"""Clean up subprocess"""
pass
```
### 3. **Factory Pattern - Strategy Selection**
Automatically select the best integration strategy for each tool.
```python
# patterns/factory.py
from typing import Type, Dict, List
import inspect
class MiddlewareStrategyFactory:
"""Factory for selecting optimal integration strategy"""
strategy_registry: Dict[IntegrationStrategy, Type[IntegrationHandler]] = {
IntegrationStrategy.LOGGER_REPLACEMENT: LoggerReplacementStrategy,
IntegrationStrategy.OUTPUT_CAPTURE: OutputCaptureStrategy,
IntegrationStrategy.SUBPROCESS_WRAPPER: SubprocessWrapperStrategy,
}
@classmethod
def select_strategy(cls, tool_instance: Any) -> IntegrationStrategy:
"""Automatically select best strategy for tool"""
# Check for pluggable logger interface
if cls._has_logger_interface(tool_instance):
return IntegrationStrategy.LOGGER_REPLACEMENT
# Check for direct API hooks
if cls._has_api_hooks(tool_instance):
return IntegrationStrategy.API_HOOKS
# Check if tool is a module vs executable
if cls._is_subprocess_tool(tool_instance):
return IntegrationStrategy.SUBPROCESS_WRAPPER
# Default to output capture
return IntegrationStrategy.OUTPUT_CAPTURE
@classmethod
def create_handler(cls, strategy: IntegrationStrategy) -> IntegrationHandler:
"""Create handler for selected strategy"""
handler_class = cls.strategy_registry[strategy]
return handler_class()
@classmethod
def _has_logger_interface(cls, tool_instance: Any) -> bool:
"""Check if tool has replaceable logger"""
# Look for common logger patterns
logger_indicators = [
'logger', 'log', 'set_logger', 'get_logger',
'_logger', 'logging', 'verbose', 'quiet'
]
for attr in logger_indicators:
if hasattr(tool_instance, attr):
return True
# Check for logging module usage
if hasattr(tool_instance, '__module__'):
try:
module = inspect.getmodule(tool_instance)
return 'logging' in str(module.__dict__)
except:
pass
return False
@classmethod
def _has_api_hooks(cls, tool_instance: Any) -> bool:
"""Check if tool provides direct API hooks"""
hook_indicators = [
'add_hook', 'register_callback', 'set_callback',
'on_progress', 'on_complete', 'on_error'
]
return any(hasattr(tool_instance, attr) for attr in hook_indicators)
@classmethod
def _is_subprocess_tool(cls, tool_instance: Any) -> bool:
"""Check if tool should be wrapped as subprocess"""
# Check if it's a string (command name) or Path
return isinstance(tool_instance, (str, Path))
```
### 4. **Observer Pattern - Event Broadcasting**
Broadcast tool events to multiple MCP contexts or handlers.
```python
# patterns/observer.py
from typing import List, Set, Callable, Any
from dataclasses import dataclass
from enum import Enum
class ToolEvent(Enum):
OPERATION_START = "operation_start"
OPERATION_COMPLETE = "operation_complete"
OPERATION_ERROR = "operation_error"
PROGRESS_UPDATE = "progress_update"
USER_INTERACTION = "user_interaction"
LOG_MESSAGE = "log_message"
@dataclass
class ToolEventData:
event_type: ToolEvent
operation_id: str
data: Dict[str, Any]
timestamp: float
context: Optional[Any] = None
class ToolEventObserver(ABC):
"""Abstract observer for tool events"""
@abstractmethod
async def handle_event(self, event: ToolEventData) -> None:
"""Handle tool event"""
pass
class MCPEventObserver(ToolEventObserver):
"""MCP-specific event observer"""
def __init__(self, context: Context):
self.context = context
async def handle_event(self, event: ToolEventData) -> None:
"""Translate tool events to MCP context calls"""
event_handlers = {
ToolEvent.OPERATION_START: self._handle_operation_start,
ToolEvent.OPERATION_COMPLETE: self._handle_operation_complete,
ToolEvent.OPERATION_ERROR: self._handle_operation_error,
ToolEvent.PROGRESS_UPDATE: self._handle_progress_update,
ToolEvent.USER_INTERACTION: self._handle_user_interaction,
ToolEvent.LOG_MESSAGE: self._handle_log_message,
}
handler = event_handlers.get(event.event_type)
if handler:
await handler(event)
async def _handle_operation_start(self, event: ToolEventData) -> None:
await self.context.log(
level='info',
message=f"🔄 Started: {event.data.get('operation_name', 'Unknown operation')}"
)
async def _handle_operation_complete(self, event: ToolEventData) -> None:
await self.context.log(
level='info',
message=f"✅ Completed: {event.data.get('operation_name', 'Operation')}"
)
async def _handle_operation_error(self, event: ToolEventData) -> None:
error_msg = event.data.get('error_message', 'Unknown error')
await self.context.log(level='error', message=f"❌ Error: {error_msg}")
async def _handle_progress_update(self, event: ToolEventData) -> None:
if hasattr(self.context, 'progress'):
await self.context.progress(
operation_id=event.operation_id,
progress=event.data.get('progress', 0),
total=event.data.get('total', 100),
current=event.data.get('current', 0),
message=event.data.get('message', '')
)
async def _handle_user_interaction(self, event: ToolEventData) -> None:
if hasattr(self.context, 'request_user_input'):
response = await self.context.request_user_input(
prompt=event.data.get('prompt', 'Confirmation required'),
input_type=event.data.get('input_type', 'confirmation')
)
# Store response in event for tool to access
event.data['user_response'] = response
async def _handle_log_message(self, event: ToolEventData) -> None:
await self.context.log(
level=event.data.get('level', 'info'),
message=event.data.get('message', '')
)
class ToolEventBroadcaster:
"""Broadcasts tool events to registered observers"""
def __init__(self):
self.observers: Set[ToolEventObserver] = set()
def add_observer(self, observer: ToolEventObserver) -> None:
"""Add event observer"""
self.observers.add(observer)
def remove_observer(self, observer: ToolEventObserver) -> None:
"""Remove event observer"""
self.observers.discard(observer)
async def broadcast_event(self, event: ToolEventData) -> None:
"""Broadcast event to all observers"""
tasks = [observer.handle_event(event) for observer in self.observers]
await asyncio.gather(*tasks, return_exceptions=True)
async def emit_operation_start(self, operation_id: str, operation_name: str) -> None:
"""Emit operation start event"""
event = ToolEventData(
event_type=ToolEvent.OPERATION_START,
operation_id=operation_id,
data={'operation_name': operation_name},
timestamp=time.time()
)
await self.broadcast_event(event)
async def emit_progress(
self,
operation_id: str,
progress: float,
total: int,
current: int,
message: str = ""
) -> None:
"""Emit progress update event"""
event = ToolEventData(
event_type=ToolEvent.PROGRESS_UPDATE,
operation_id=operation_id,
data={
'progress': progress,
'total': total,
'current': current,
'message': message
},
timestamp=time.time()
)
await self.broadcast_event(event)
```
### 5. **Decorator Pattern - Middleware Application**
Apply middleware through decorators for clean integration.
```python
# patterns/decorator.py
from functools import wraps
from typing import Callable, Any, TypeVar, ParamSpec
P = ParamSpec('P')
R = TypeVar('R')
def with_mcp_middleware(
tool_name: str,
integration_strategy: Optional[IntegrationStrategy] = None,
require_confirmation: bool = False,
enable_progress: bool = True
):
"""Decorator to apply MCP middleware to tool operations"""
def decorator(func: Callable[P, R]) -> Callable[P, R]:
@wraps(func)
async def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
# Extract context from args/kwargs
context = kwargs.get('context') or args[0] if args else None
if not isinstance(context, Context):
raise ValueError("MCP context required for middleware")
# Create operation ID
operation_id = f"{func.__name__}_{int(time.time())}"
# Create middleware
middleware = MiddlewareFactory.create_middleware(
tool_name, context, operation_id
)
# Select integration strategy
if integration_strategy is None:
strategy = MiddlewareStrategyFactory.select_strategy(tool_name)
else:
strategy = integration_strategy
# Create handler
handler = MiddlewareStrategyFactory.create_handler(strategy)
# Apply middleware
try:
await handler.integrate(tool_name, middleware)
# Configure middleware options
if hasattr(middleware, 'set_require_confirmation'):
middleware.set_require_confirmation(require_confirmation)
if hasattr(middleware, 'set_enable_progress'):
middleware.set_enable_progress(enable_progress)
# Execute original function
result = await func(*args, **kwargs)
return result
finally:
# Clean up middleware
await handler.cleanup(tool_name)
return wrapper
return decorator
# Usage example:
@with_mcp_middleware('esptool', require_confirmation=True)
async def flash_esp32(context: Context, port: str, firmware: str) -> str:
"""Flash ESP32 with automatic middleware integration"""
# All esptool operations in this function now use MCP middleware
with detect_chip(port) as esp:
write_flash(esp, [(0x1000, firmware)])
return "Flashing completed"
```
### 6. **Template Method Pattern - Common Integration Flow**
Define standard integration workflow with customizable steps.
```python
# patterns/template_method.py
class MiddlewareIntegrationTemplate:
"""Template for middleware integration workflow"""
async def integrate_tool(
self,
tool_instance: Any,
context: Context,
operation_id: str
) -> Any:
"""Template method for tool integration"""
# Step 1: Analyze tool capabilities
capabilities = await self.analyze_tool(tool_instance)
# Step 2: Select integration strategy
strategy = await self.select_strategy(tool_instance, capabilities)
# Step 3: Create middleware
middleware = await self.create_middleware(context, operation_id, capabilities)
# Step 4: Install hooks
await self.install_hooks(tool_instance, middleware, strategy)
# Step 5: Configure options
await self.configure_middleware(middleware, capabilities)
# Step 6: Start monitoring
await self.start_monitoring(tool_instance, middleware)
return middleware
@abstractmethod
async def analyze_tool(self, tool_instance: Any) -> Dict[str, Any]:
"""Analyze tool's capabilities and interfaces"""
pass
@abstractmethod
async def select_strategy(
self,
tool_instance: Any,
capabilities: Dict[str, Any]
) -> IntegrationStrategy:
"""Select appropriate integration strategy"""
pass
async def create_middleware(
self,
context: Context,
operation_id: str,
capabilities: Dict[str, Any]
) -> ToolAdapter:
"""Create middleware instance (default implementation)"""
return MiddlewareFactory.create_middleware(
self.tool_name, context, operation_id
)
@abstractmethod
async def install_hooks(
self,
tool_instance: Any,
middleware: ToolAdapter,
strategy: IntegrationStrategy
) -> None:
"""Install middleware hooks"""
pass
async def configure_middleware(
self,
middleware: ToolAdapter,
capabilities: Dict[str, Any]
) -> None:
"""Configure middleware options (default implementation)"""
# Configure based on detected capabilities
if capabilities.get('supports_progress', False):
middleware.enable_progress_tracking()
if capabilities.get('has_interactive_operations', False):
middleware.enable_user_confirmations()
async def start_monitoring(
self,
tool_instance: Any,
middleware: ToolAdapter
) -> None:
"""Start monitoring tool operations (default implementation)"""
# Set up event monitoring if supported
if hasattr(middleware, 'start_event_monitoring'):
await middleware.start_event_monitoring()
```
## 🎯 Pattern Composition Example
### Complete ESPTool Middleware Using All Patterns
```python
# esptool_complete_middleware.py
class ESPToolCompleteMiddleware(MiddlewareIntegrationTemplate):
"""Complete ESPTool middleware using all design patterns"""
def __init__(self):
self.tool_name = 'esptool'
self.event_broadcaster = ToolEventBroadcaster()
async def analyze_tool(self, tool_instance: Any) -> Dict[str, Any]:
"""Analyze esptool capabilities"""
return {
'has_logger_interface': True,
'supports_progress': True,
'has_interactive_operations': True,
'logger_module': 'esptool.logger',
'progress_method': 'progress_bar',
'critical_operations': [
'erase_flash', 'burn_efuse', 'enable_secure_boot'
]
}
async def select_strategy(
self,
tool_instance: Any,
capabilities: Dict[str, Any]
) -> IntegrationStrategy:
"""Select logger replacement strategy for esptool"""
return IntegrationStrategy.LOGGER_REPLACEMENT
async def install_hooks(
self,
tool_instance: Any,
middleware: ToolAdapter,
strategy: IntegrationStrategy
) -> None:
"""Install esptool-specific hooks"""
# Add MCP observer to event broadcaster
mcp_observer = MCPEventObserver(middleware.context)
self.event_broadcaster.add_observer(mcp_observer)
# Install logger replacement
handler = MiddlewareStrategyFactory.create_handler(strategy)
await handler.integrate(tool_instance, middleware)
# Set up event broadcasting in middleware
middleware.set_event_broadcaster(self.event_broadcaster)
# Usage with decorator:
@with_mcp_middleware('esptool')
async def advanced_esp_operation(context: Context, config: Dict) -> str:
"""Advanced ESP operation with complete middleware"""
# All patterns working together:
# - Adapter pattern handles esptool interface
# - Strategy pattern selects logger replacement
# - Observer pattern broadcasts events
# - Factory pattern creates appropriate handlers
# - Template method ensures consistent integration
# - Decorator pattern applies everything transparently
result = await perform_complex_esp_operation(config)
return result
```
These reusable patterns provide a comprehensive framework for integrating any CLI tool with MCP servers, ensuring consistent, maintainable, and extensible middleware implementations.
Reference in New Issue View Git Blame Copy Permalink