diff --git a/README.md b/README.md index 1d52ad9..39869a1 100644 --- a/README.md +++ b/README.md @@ -1155,7 +1155,7 @@ mqtt-{subdomain}.yourdomain.com { ### 🔧 Docker Compose Integration -**Complete Stack with Wildcard Certificates:** +**Complete Stack with caddy-docker-proxy:** ```yaml # docker-compose.yml services: @@ -1167,41 +1167,387 @@ services: networks: - caddy labels: - # Dynamic hostname routing with wildcard cert - caddy: "*.mqtt.yourdomain.com" - caddy.tls: "*.yourdomain.com" - caddy.tls.dns: cloudflare + # caddy-docker-proxy automatically adds to Caddyfile + caddy: mqtt-control.yourdomain.com caddy.reverse_proxy: "{{upstreams 3000}}" - - # Route based on broker ID in hostname - caddy.handle_path: "/broker/*" - caddy.handle_path.reverse_proxy: "localhost:{dynamic_port}" + caddy.tls: internal # or use DNS challenge for public - caddy: - image: caddy:2-alpine - ports: - - "80:80" - - "443:443" - volumes: - - caddy_data:/data - - caddy_config:/config - - ./Caddyfile:/etc/caddy/Caddyfile + # Example spawned broker container (created by mcmqtt) + mqtt-broker-customer-123: + image: python:3.11-slim + command: uvx amqtt --config /broker.yaml networks: - caddy - environment: - - CLOUDFLARE_API_TOKEN=${CLOUDFLARE_TOKEN} labels: - caddy_controlled_server: "" + # Automatic HTTPS routing via caddy-docker-proxy + caddy: customer-123.mqtt.yourdomain.com + caddy.reverse_proxy: "{{upstreams 1883}}" + caddy.tls.dns: cloudflare + # WebSocket support + caddy.handle_path: "/ws" + caddy.handle_path.reverse_proxy: "{{upstreams 9001}}" + caddy.@websocket.header: "Connection *Upgrade*" + caddy.@websocket.header_2: "Upgrade websocket" networks: caddy: - external: true - -volumes: - caddy_data: - caddy_config: + external: true # Managed by load-balancers/caddy ``` +**The Magic: Just Labels = Instant HTTPS Routing!** +```bash +# When mcmqtt spawns a broker, it adds these labels: +{ + "tool": "mqtt_spawn_broker", + "arguments": { + "name": "api-coordination", + "websocket_port": 9001, + "caddy_labels": { + "caddy": "api-coord.mqtt.yourdomain.com", + "caddy.reverse_proxy": "{{upstreams 1883}}", + "caddy.handle_path": "/ws", + "caddy.handle_path.reverse_proxy": "{{upstreams 9001}}", + "caddy.tls.dns": "cloudflare" + } + } +} + +# caddy-docker-proxy automatically detects new container +# Adds route to Caddyfile: api-coord.mqtt.yourdomain.com +# Issues wildcard certificate via Cloudflare DNS +# Result: Instant HTTPS routing with zero manual config! +``` + +### 🐳 Docker Socket Magic = Instant Infrastructure + +**How mcmqtt + caddy-docker-proxy Works:** + +1. **mcmqtt has Docker socket access** (`/var/run/docker.sock` mapped) +2. **Spawn broker = Spawn Docker container** with caddy labels +3. **caddy-docker-proxy detects new container** instantly +4. **Automatic Caddyfile update** with HTTPS routing +5. **Wildcard certificate covers any subdomain** pattern + +**Implementation Flow:** +```bash +# User calls mqtt_spawn_broker tool +{ + "tool": "mqtt_spawn_broker", + "arguments": { + "name": "customer-acme", + "websocket_port": 9001, + "public_hostname": "acme.mqtt.yourapp.com" + } +} + +# mcmqtt executes (behind the scenes): +docker run -d \ + --name mqtt-broker-customer-acme \ + --network caddy \ + -p 1883 -p 9001 \ + --label "caddy=acme.mqtt.yourapp.com" \ + --label "caddy.reverse_proxy={{upstreams 1883}}" \ + --label "caddy.handle_path=/ws" \ + --label "caddy.handle_path.reverse_proxy={{upstreams 9001}}" \ + --label "caddy.tls.dns=cloudflare" \ + python:3.11-slim uvx amqtt + +# caddy-docker-proxy immediately detects container +# Auto-generates Caddyfile entry: +# acme.mqtt.yourapp.com { +# reverse_proxy mqtt-broker-customer-acme:1883 +# handle_path /ws { +# reverse_proxy mqtt-broker-customer-acme:9001 +# } +# tls { +# dns cloudflare +# } +# } + +# RESULT: https://acme.mqtt.yourapp.com is LIVE! +``` + +**Zero Configuration Infrastructure:** +- ✅ **No manual Caddyfile editing** +- ✅ **No certificate management** +- ✅ **No DNS configuration** (wildcard covers all) +- ✅ **No load balancer setup** +- ✅ **No container orchestration** +- ✅ **No SSL/TLS configuration** + +**Just call tool → Get production HTTPS endpoint!** + +### 🌍 Public Internet Access = Global Coordination + +**caddy-docker-proxy makes brokers PUBLICLY accessible:** + +```bash +# Before: Local-only broker +localhost:1883 # Only accessible from same machine + +# After: Public HTTPS endpoint +https://customer-acme.mqtt.yourapp.com # Accessible from ANYWHERE! +wss://customer-acme.mqtt.yourapp.com/ws # WebSocket from browsers +``` + +**Global Accessibility Examples:** +```bash +# Mobile apps connect from anywhere +{ + "broker_url": "wss://mobile-sync.mqtt.yourapp.com/ws", + "auth": {"username": "mobile_app", "password": "secure_token"} +} + +# IoT devices in different countries +{ + "broker_url": "mqtts://iot-fleet.mqtt.yourapp.com:8883", + "client_id": "device_12345_tokyo", + "clean_session": true +} + +# Third-party integrations +{ + "webhook_endpoint": "https://webhooks.mqtt.yourapp.com/stripe", + "mqtt_publish": "payments/stripe/events" +} + +# Remote agent coordination +{ + "coordinator": "wss://agents.mqtt.yourapp.com/ws", + "agent_id": "data_processor_eu_west", + "capabilities": ["pdf_analysis", "ocr", "translation"] +} +``` + +**Production Security Features:** +- 🔒 **Automatic HTTPS/TLS encryption** via Let's Encrypt +- 🛡️ **DDoS protection** via Cloudflare/Caddy +- 🔑 **Authentication** built into MQTT broker +- 🌐 **Global CDN** if using Cloudflare +- 📊 **Request logging & monitoring** via Caddy +- ⚡ **Rate limiting** and traffic shaping +- 🔄 **Automatic failover** with health checks + +**Use Cases Unlocked:** +- **Global IoT fleets** with secure public endpoints +- **Mobile app synchronization** via WebSocket +- **Third-party webhook integration** with MQTT publishing +- **Remote agent coordination** across different cloud providers +- **Public API backends** with real-time MQTT coordination +- **Multi-region deployment** with geographic routing + +**The Revolutionary Part**: Transform any `mqtt_spawn_broker` call into a **globally accessible, production-ready MQTT endpoint** with enterprise-grade security - all with zero manual infrastructure setup! + +### 🌐 External Agent Integration = Unlimited Possibilities + +**Outside agents can now join your coordination network with valid certificates!** + +**Before mcmqtt + Caddy:** +```bash +# Local agents only - no external connectivity +claude-agent-1 ←→ local MQTT ←→ claude-agent-2 + ↕ + (isolated) +``` + +**After mcmqtt + Caddy:** +```bash +# Global agent network with secure internet connectivity +claude-agent-1 ←→ wss://coord.mqtt.yourapp.com ←→ claude-agent-2 + ↕ +mobile-app ←→ external-api ←→ iot-devices ←→ remote-agents + ↕ + third-party-services ←→ webhook-integrations +``` + +**External Agent Connection Examples:** + +**1. Remote AI Agents (Different Servers/Providers):** +```bash +# Agent running on AWS connects to your broker +{ + "connection": { + "url": "wss://ai-coordination.mqtt.yourapp.com/ws", + "client_id": "aws-analysis-agent-east-1", + "username": "external_agent", + "password": "secure_api_key_2025" + }, + "capabilities": ["data_analysis", "ml_inference", "report_generation"] +} + +# Agent running on Google Cloud joins the same network +{ + "connection": { + "url": "wss://ai-coordination.mqtt.yourapp.com/ws", + "client_id": "gcp-processing-agent-europe", + "username": "external_agent", + "password": "secure_api_key_2025" + }, + "capabilities": ["image_processing", "ocr", "translation"] +} +``` + +**2. Mobile Apps & Web Clients:** +```javascript +// JavaScript web client connects with valid certificate +const mqtt = new Paho.MQTT.Client("ai-coordination.mqtt.yourapp.com", 443, "web-client-" + Date.now()); + +mqtt.connect({ + useSSL: true, // Automatic certificate validation + userName: "web_client", + password: "secure_web_token", + onSuccess: () => { + // Subscribe to agent coordination topics + mqtt.subscribe("agents/+/status"); + mqtt.subscribe("tasks/web/+"); + } +}); +``` + +**3. Third-Party Service Integration:** +```bash +# Zapier/IFTTT webhook connects to your agent network +POST https://webhook-bridge.mqtt.yourapp.com/zapier +{ + "trigger": "new_email", + "data": { + "subject": "Customer Support Request", + "sender": "customer@example.com", + "content": "Need help with billing" + }, + "mqtt_topic": "support/requests/incoming" +} + +# Your agents receive the event and coordinate response +``` + +**4. IoT Device Fleet (Global):** +```python +# IoT device in factory connects from anywhere +import paho.mqtt.client as mqtt + +client = mqtt.Client("factory_sensor_tokyo_001") +client.username_pw_set("iot_device", "factory_secure_2025") + +# Valid certificate, no certificate warnings! +client.tls_set() # Automatic certificate validation +client.connect("iot-fleet.mqtt.yourapp.com", 8883, 60) + +# Publish sensor data to global coordination network +client.publish("sensors/tokyo/temperature", "23.5") +``` + +**5. Multi-Organization Coordination:** +```bash +# Partner company agents join your coordination network +{ + "partner_agent": { + "url": "wss://partner-coordination.mqtt.yourapp.com/ws", + "client_id": "partner_company_agent_001", + "auth": "partner_api_key_secure", + "permissions": ["read_public_topics", "publish_partner_results"] + } +} + +# Secure multi-tenant coordination with proper isolation +``` + +**Key Benefits for External Connectivity:** + +🌍 **Global Reach**: Agents anywhere on internet can join coordination +🔒 **Valid Certificates**: No certificate warnings or security bypasses +⚡ **Instant Deployment**: Spawn broker → Get internet-routable URL +🔑 **Secure Authentication**: Built-in MQTT auth prevents unauthorized access +🚀 **Zero Infrastructure**: No VPN, firewall rules, or DNS management +📱 **Browser Compatible**: WebSocket support for web/mobile clients +🔄 **Auto-Scaling**: Spawn more brokers as coordination needs grow +🛡️ **Production Ready**: DDoS protection, rate limiting, monitoring included + +**The Game Changer**: Turn any local agent coordination into a **globally accessible, secure coordination platform** that external agents, mobile apps, IoT devices, and third-party services can securely join - all with a single tool call! + +### 🚀 Like ngrok, but for MQTT Coordination (and Better!) + +**ngrok vs mcmqtt + Caddy Comparison:** + +| Feature | ngrok | mcmqtt + Caddy | +|---------|-------|----------------| +| **Purpose** | Expose local HTTP servers | Spawn + expose MQTT brokers | +| **URL Format** | `random-id.ngrok.io` | `your-domain.mqtt.yourapp.com` | +| **Certificate** | ngrok managed | Your own domain wildcard | +| **Persistence** | Tunnels die when process stops | Production infrastructure | +| **Scaling** | One tunnel per process | Unlimited brokers per domain | +| **Cost** | $$$ for custom domains | Free with your domain | +| **Protocol Support** | HTTP/HTTPS primarily | MQTT, WebSocket, HTTP | +| **Authentication** | Basic HTTP auth | MQTT auth + Caddy auth | +| **Production Ready** | Development tool | Enterprise infrastructure | + +**What mcmqtt + Caddy Does Better:** + +```bash +# ngrok style: Temporary tunnel to existing service +ngrok http 3000 # → https://abc123.ngrok.io (random URL, dies when stopped) + +# mcmqtt style: Spawn service WITH permanent public endpoint +{ + "tool": "mqtt_spawn_broker", + "arguments": { + "name": "customer-demo", + "public_hostname": "demo.mqtt.yourapp.com" # Your domain, permanent + } +} +# → wss://demo.mqtt.yourapp.com/ws (YOUR domain, persists) +``` + +**The mcmqtt Advantage:** + +🏗️ **Infrastructure Creation**: Spawns the service AND the public endpoint +🌐 **Your Domain**: Use your branded domain, not random subdomains +💎 **Permanent**: URLs persist, perfect for production use +🔄 **Multi-Protocol**: MQTT, WebSocket, HTTP all supported +📈 **Unlimited Scale**: Spawn hundreds of brokers on same domain +🔒 **Enterprise Security**: Your certificates, your control +💰 **Cost Effective**: No per-tunnel fees, just your domain costs +🎯 **Purpose Built**: Designed specifically for agent coordination + +**Real-World Comparison:** + +**Development with ngrok:** +```bash +# Start local MQTT broker +mosquitto -p 1883 + +# Expose via ngrok (HTTP only, need wrapper) +ngrok http 1883 # Doesn't actually work for MQTT! + +# Or use ngrok TCP tunnel +ngrok tcp 1883 # → tcp://0.tcp.ngrok.io:12345 (ugly, temporary) +``` + +**Production with mcmqtt + Caddy:** +```bash +# Spawn broker with public endpoint in one call +{ + "tool": "mqtt_spawn_broker", + "arguments": { + "name": "production-api", + "public_hostname": "api.mqtt.yourapp.com", + "websocket_port": 9001, + "auth_required": true + } +} + +# Result: +# - MQTT: mqtts://api.mqtt.yourapp.com:8883 +# - WebSocket: wss://api.mqtt.yourapp.com/ws +# - Both with valid certificates, permanent URLs +``` + +**The "Instant Global Infrastructure" Revolution:** +- **ngrok**: "Make my local thing temporarily accessible" +- **mcmqtt + Caddy**: "Create production infrastructure that happens to be local" + +This isn't just exposing existing services - it's **spawning infrastructure with global access built-in!** + **Advanced Caddyfile with Wildcard DNS:** ```caddyfile # Global options for wildcard certificate management