--- title: "Configuration Reference" weight: 30 description: "Complete technical reference for Docker configurations, environment variables, and build parameters" --- # Docker Configuration Reference This reference provides complete technical specifications for all Docker configurations, environment variables, build parameters, and service definitions in the Flamenco development environment. ## File Structure ``` flamenco/ ├── Dockerfile.dev # Multi-stage Docker build configuration ├── compose.dev.yml # Docker Compose service definitions ├── Makefile.docker # Management commands ├── .env # Environment variable configuration └── .env.dev # Default environment template ``` ## Dockerfile.dev Stages ### Base Stage ```dockerfile FROM golang:1.24-alpine AS base ``` **System packages installed**: - `git` - Version control operations - `make` - Build automation - `nodejs`, `npm`, `yarn` - JavaScript tooling - `openjdk11-jre-headless` - Java runtime for code generators - `sqlite` - Database engine - `bash`, `curl`, `ca-certificates` - System utilities - `python3`, `python3-dev`, `py3-pip` - Python development **Environment variables**: - `CGO_ENABLED=0` - Pure Go builds - `GOPROXY=https://proxy.golang.org,direct` - Go module proxy with fallback - `GOSUMDB=sum.golang.org` - Go module checksum verification ### Dependencies Stage ```dockerfile FROM base AS deps ``` **Operations**: 1. Copy dependency manifests (`go.mod`, `go.sum`, `package.json`, `yarn.lock`, `pyproject.toml`) 2. Download Go modules via proxy 3. Install Node.js dependencies with frozen lockfile 4. Install Python uv package manager 5. Sync Python dependencies (development packages excluded) **Caching strategy**: Dependencies cached in separate layer from source code. ### Build-tools Stage ```dockerfile FROM deps AS build-tools ``` **Operations**: 1. Copy full source code 2. Compile Mage build tool 3. Install Flamenco code generators 4. Prepare build environment **Generated tools**: - `./mage` - Flamenco build automation - Code generation tools (OpenAPI, mock generation) ### Development Stage ```dockerfile FROM build-tools AS development ``` **Additional tools installed**: - `github.com/air-verse/air@v1.52.3` - Go hot-reloading - `github.com/githubnemo/CompileDaemon@latest` - Alternative hot-reloading **Build operations**: 1. Generate API code and mocks 2. Build static web assets 3. Compile Flamenco binaries 4. Copy binaries to `/usr/local/bin/` (avoids mount conflicts) **Exposed ports**: - `8080` - Manager API and web interface - `8081` - Vue.js development server - `8082` - pprof profiling endpoint ### Production Stage ```dockerfile FROM alpine:latest AS production ``` **Minimal runtime dependencies**: - `ca-certificates` - TLS certificate validation - `sqlite` - Database engine - `tzdata` - Timezone information **Security**: - Non-root user `flamenco` (UID/GID 1000) - Restricted file permissions - Minimal attack surface **Data directories**: - `/app` - Application binaries - `/data` - Database and persistent data - `/shared-storage` - Render farm shared storage ### Test Stage ```dockerfile FROM build-tools AS test ``` **Purpose**: Continuous integration testing **Operations**: Code generation and test execution ### Tools Stage ```dockerfile FROM base AS tools ``` **Development tools**: - `golangci-lint` - Go linting - `goose` - Database migrations - `sqlc` - SQL code generation ## Docker Compose Configuration ### Service Definitions #### flamenco-manager **Base configuration**: ```yaml build: context: . dockerfile: Dockerfile.dev target: development container_name: ${COMPOSE_PROJECT_NAME}-manager hostname: flamenco-manager ``` **Port mappings**: - `${MANAGER_PORT:-8080}:8080` - Main API and web interface - `${MANAGER_DEBUG_PORT:-8082}:8082` - pprof debugging endpoint **Volume mounts**: - `.:/app` - Source code (development) - `/app/node_modules` - Prevent override - `/app/web/app/node_modules` - Prevent override - `flamenco-data:/data` - Database persistence - `flamenco-shared:/shared-storage` - Shared render storage - `go-mod-cache:/go/pkg/mod` - Go module cache - `yarn-cache:/usr/local/share/.cache/yarn` - Yarn package cache **Environment variables**: See [Environment Variables](#environment-variables) section. **Health check**: ```yaml test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:8080/api/v3/version"] interval: 30s timeout: 10s retries: 3 start_period: 40s ``` #### flamenco-worker **Base configuration**: ```yaml build: context: . dockerfile: Dockerfile.dev target: development container_name: ${COMPOSE_PROJECT_NAME}-worker hostname: flamenco-worker ``` **Volume mounts**: - `.:/app` - Source code (development) - `/app/node_modules` - Prevent override - `flamenco-shared:/shared-storage` - Shared render storage - `worker-data:/data` - Worker database - `go-mod-cache:/go/pkg/mod` - Go module cache **Dependencies**: Requires `flamenco-manager` to be running. ### Development Services #### webapp-dev Vue.js development server with hot-reloading. **Configuration**: - Port: `${WEBAPP_DEV_PORT:-8081}:8081` - Command: `yarn dev --host 0.0.0.0` - Working directory: `/app/web/app` - Profile: `dev-tools` #### docs-dev Hugo documentation development server. **Configuration**: - Port: `${DOCS_DEV_PORT:-1313}:1313` - Working directory: `/app/web/project-website` - Hugo version: `v0.121.2` - Profile: `dev-tools` #### dev-tools Database management and development utilities. **Configuration**: - Target: `tools` (Dockerfile stage) - Available tools: goose, sqlc, golangci-lint - Profile: `dev-tools` ### Networks #### flamenco-net Internal bridge network for service communication. **Configuration**: ```yaml driver: bridge name: ${COMPOSE_PROJECT_NAME}-network ``` #### caddy (External) External network for reverse proxy integration. **Usage**: Supports automatic HTTPS with Caddy labels. ### Volumes #### Persistent Data Volumes - `flamenco-data` - Manager database and configuration - `flamenco-shared` - Shared storage for renders and assets - `worker-data` - Worker database and state #### Development Cache Volumes - `go-mod-cache` - Go module download cache - `yarn-cache` - Node.js package cache **Benefits**: Significantly reduces rebuild times by preserving dependency caches. ## Environment Variables ### Project Configuration | Variable | Default | Description | |----------|---------|-------------| | `COMPOSE_PROJECT_NAME` | `flamenco-dev` | Docker Compose project prefix | | `DOMAIN` | `flamenco.l.supported.systems` | Base domain for reverse proxy | ### Service Ports | Variable | Default | Description | |----------|---------|-------------| | `MANAGER_PORT` | `9000` | Manager API and web interface | | `MANAGER_DEBUG_PORT` | `8082` | pprof debugging endpoint | | `WEBAPP_DEV_PORT` | `8081` | Vue.js development server | | `DOCS_DEV_PORT` | `1313` | Hugo documentation server | ### Manager Configuration | Variable | Default | Description | |----------|---------|-------------| | `MANAGER_HOST` | `0.0.0.0` | Network binding interface | | `LOG_LEVEL` | `debug` | Logging verbosity level | | `DATABASE_CHECK_PERIOD` | `1m` | Database health check interval | | `ENABLE_PPROF` | `true` | Performance profiling endpoint | | `DATABASE_FILE` | `/data/flamenco-manager.sqlite` | Database file path | | `SHARED_STORAGE_PATH` | `/shared-storage` | Shared storage mount point | ### Worker Configuration | Variable | Default | Description | |----------|---------|-------------| | `WORKER_NAME` | `docker-dev-worker` | Worker identification name | | `WORKER_TAGS` | `docker,development,local` | Worker capability tags | | `TASK_TIMEOUT` | `10m` | Maximum task execution time | | `WORKER_SLEEP_SCHEDULE` | _(empty)_ | Worker sleep periods (HH:MM-HH:MM) | | `MANAGER_URL` | `http://flamenco-manager:8080` | Manager API endpoint | ### Platform Variables Multi-platform executable paths for render farm heterogeneous environments: #### Blender Paths | Variable | Default | Platform | |----------|---------|----------| | `BLENDER_LINUX` | `/usr/local/blender/blender` | Linux | | `BLENDER_WINDOWS` | `C:\Program Files\Blender Foundation\Blender\blender.exe` | Windows | | `BLENDER_DARWIN` | `/Applications/Blender.app/Contents/MacOS/Blender` | macOS | #### FFmpeg Paths | Variable | Default | Platform | |----------|---------|----------| | `FFMPEG_LINUX` | `/usr/bin/ffmpeg` | Linux | | `FFMPEG_WINDOWS` | `C:\ffmpeg\bin\ffmpeg.exe` | Windows | | `FFMPEG_DARWIN` | `/usr/local/bin/ffmpeg` | macOS | ### Shaman Configuration | Variable | Default | Description | |----------|---------|-------------| | `SHAMAN_ENABLED` | `false` | Enable Shaman asset management | | `SHAMAN_CHECKOUT_PATH` | `/shared-storage/shaman-checkouts` | Asset checkout directory | | `SHAMAN_STORAGE_PATH` | `/data/shaman-storage` | Asset storage directory | ### Development Options | Variable | Default | Description | |----------|---------|-------------| | `ENVIRONMENT` | `development` | Environment marker | | `DEV_MODE` | `true` | Enable development features | | `CORS_ALLOW_ALL` | `true` | Allow all CORS origins (dev only) | | `DISABLE_AUTH` | `true` | Disable authentication (dev only) | | `DEBUG_ENDPOINTS` | `true` | Enable debug API endpoints | ### Resource Limits | Variable | Default | Description | |----------|---------|-------------| | `MANAGER_MEMORY_LIMIT` | `1g` | Manager container memory limit | | `WORKER_MEMORY_LIMIT` | `512m` | Worker container memory limit | | `WORKER_REPLICAS` | `1` | Number of worker instances | ### External Services | Variable | Default | Description | |----------|---------|-------------| | `MQTT_ENABLED` | `false` | Enable MQTT messaging | | `MQTT_BROKER` | `mqtt://localhost:1883` | MQTT broker URL | | `MQTT_USERNAME` | _(empty)_ | MQTT authentication username | | `MQTT_PASSWORD` | _(empty)_ | MQTT authentication password | ## Makefile.docker Targets ### Setup Commands | Target | Description | |--------|-------------| | `help` | Display available targets and current configuration | | `prerequisites` | Verify Docker and Docker Compose installation | | `network-setup` | Create external Docker networks | | `env-setup` | Copy `.env.dev` to `.env` if not exists | | `dev-setup` | Complete development environment setup | ### Development Commands | Target | Description | |--------|-------------| | `dev-start` | Start core services (Manager + Worker) | | `dev-tools` | Start development tools (Vue.js, Hugo, profiling) | | `dev-all` | Start all services including tools | | `up` | Alias for `dev-start` | ### Management Commands | Target | Description | |--------|-------------| | `status` | Display service status | | `ps` | Alias for `status` | | `logs` | Show recent logs from all services | | `logs-follow` | Follow logs from all services | | `logs-manager` | Show Manager service logs | | `logs-worker` | Show Worker service logs | ### Utility Commands | Target | Description | |--------|-------------| | `shell-manager` | Open bash shell in Manager container | | `shell-worker` | Open bash shell in Worker container | | `shell-tools` | Open bash shell in dev-tools container | | `generate` | Regenerate API code in Manager container | | `test` | Run tests in Manager container | | `webapp-build` | Build webapp static files | ### Database Commands | Target | Description | |--------|-------------| | `db-status` | Show database migration status | | `db-up` | Apply pending database migrations | | `db-down` | Rollback latest database migration | ### Control Commands | Target | Description | |--------|-------------| | `dev-stop` | Stop all services | | `down` | Alias for `dev-stop` | | `dev-restart` | Restart all services | | `dev-clean` | Stop services and remove volumes | | `dev-rebuild` | Rebuild images and restart services | ### Production Commands | Target | Description | |--------|-------------| | `prod-build` | Build production Docker images | | `prod-run` | Run production container | ### Configuration Commands | Target | Description | |--------|-------------| | `config` | Display resolved Docker Compose configuration | | `config-validate` | Validate Docker Compose file syntax | | `env-show` | Display current environment variables | ### Cleanup Commands | Target | Description | |--------|-------------| | `clean-volumes` | Remove all project volumes (destructive) | | `clean-images` | Remove project Docker images | | `clean-all` | Complete cleanup (destructive) | ### Build Commands | Target | Description | |--------|-------------| | `build` | Build development images | | `pull` | Pull latest base images | ## Performance Metrics ### Build Time Benchmarks | Component | Optimized Time | Previous Time | Improvement | |-----------|---------------|---------------|-------------| | Go modules | 21.4s | 60+ min (failure) | 168x faster | | Alpine packages | 6.8 min | ~7 min | Stable | | Python dependencies | 51.8s | Variable | 2-3x faster | | Node.js packages | 4.7s | Variable | Stable | | Total build | ~26 min | 60+ min (failure) | 100% success rate | ### Resource Requirements | Component | CPU | Memory | Storage | |-----------|-----|--------|---------| | Manager | 0.5-1 core | 512MB-1GB | 2GB | | Worker | 0.5-1 core | 256MB-512MB | 1GB | | Build process | 1-2 cores | 2GB-4GB | 5GB temporary | | Development caches | N/A | N/A | 2-3GB persistent | ### Network Performance | Operation | Bandwidth | Latency | |-----------|-----------|---------| | Go proxy downloads | ~10MB/s | <100ms | | npm registry | ~5MB/s | <200ms | | Alpine packages | ~2MB/s | <300ms | | Container registry | ~20MB/s | <50ms | ## Security Considerations ### Production Security - Non-root container execution (UID/GID 1000) - Minimal base images (Alpine Linux) - No development tools in production images - Restricted file permissions ### Development Security - **Warning**: Development configuration disables authentication - **Warning**: CORS allows all origins - **Warning**: Debug endpoints exposed - Local network binding only (0.0.0.0 inside containers) ### Secrets Management - Use environment variables for sensitive configuration - Mount secrets as files when possible - Avoid embedding secrets in images - Rotate credentials regularly ## Integration Points ### CI/CD Integration ```yaml # Example GitHub Actions integration - name: Build Docker images run: docker compose --progress plain -f compose.dev.yml build - name: Run tests run: make -f Makefile.docker test - name: Build production run: make -f Makefile.docker prod-build ``` ### Reverse Proxy Integration Labels for automatic Caddy configuration: ```yaml labels: caddy: manager.${DOMAIN} caddy.reverse_proxy: "{{upstreams 8080}}" caddy.header: "X-Forwarded-Proto https" ``` ### Monitoring Integration Endpoints for health monitoring: - `http://localhost:9000/api/v3/version` - API version - `http://localhost:9000/api/v3/status` - System status - `http://localhost:8082/debug/pprof/` - Performance profiling This reference covers all technical aspects of the Docker development environment configuration. For implementation guidance, see the [Tutorial](../tutorial/) and [Troubleshooting Guide](../how-to/).