flamenco/DOCKER_DEVELOPMENT.md

Docker Development Environment

This document describes how to set up and use the Flamenco development environment using Docker and Docker Compose.

Quick Start

Prerequisites

Ensure caddy-docker-proxy is running:

docker network create caddy
docker run -d \
  --name caddy-docker-proxy \
  --restart unless-stopped \
  --network caddy \
  -p 80:80 -p 443:443 \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  -v caddy_data:/data \
  lucaslorentz/caddy-docker-proxy:ci-alpine

Setup

1. Setup the environment:

   ./scripts/dev-setup.sh
   

2. Access Flamenco (via reverse proxy):

   • Manager: https://manager.flamenco.l.supported.systems
   • Manager API: https://manager.flamenco.l.supported.systems/api/v3

3. Access Flamenco (direct):

   • Manager Web Interface: http://localhost:8080
   • Manager API: http://localhost:8080/api/v3
   • Profiling (pprof): http://localhost:8082/debug/pprof

4. Start development tools (optional):

   ./scripts/dev-setup.sh dev-tools
   

5. Development URLs (via reverse proxy):

   • Vue.js Frontend: https://flamenco.l.supported.systems
   • Documentation: https://docs.flamenco.l.supported.systems
   • Profiling: https://profiling.flamenco.l.supported.systems

6. Development URLs (direct):

   • Vue.js Dev Server: http://localhost:8081
   • Hugo Documentation: http://localhost:1313

Architecture Overview

The Docker development environment consists of multiple stages and services:

Docker Multi-Stage Build

• base: Common dependencies (Go, Node.js, Java, etc.)
• deps: Dependencies installation and caching
• build-tools: Flamenco build tools (Mage, generators)
• development: Full development environment with hot-reloading
• builder: Production binary building
• production: Minimal runtime image
• test: Testing environment
• tools: Development utilities

Services

• flamenco-manager: Central coordination server
• flamenco-worker: Task execution daemon
• webapp-dev: Vue.js development server (hot-reloading)
• docs-dev: Hugo documentation server
• dev-tools: Database and development utilities
• shared-storage-setup: Storage initialization

Configuration

Environment Variables

Copy .env.dev to .env and customize:

cp .env.dev .env

Key configuration options:

# Project naming
COMPOSE_PROJECT_NAME=flamenco-dev

# Domain configuration for reverse proxy
DOMAIN=flamenco.l.supported.systems

# Ports (for direct access)
MANAGER_PORT=8080
WEBAPP_DEV_PORT=8081
DOCS_DEV_PORT=1313

# Development settings
LOG_LEVEL=debug
ENABLE_PPROF=true

Reverse Proxy Configuration

The environment includes caddy-docker-proxy labels for automatic SSL termination and routing:

• Manager: manager.${DOMAIN} → flamenco-manager:8080
• Vue.js Frontend: ${DOMAIN} → webapp-dev:8081
• Documentation: docs.${DOMAIN} → docs-dev:1313
• Profiling: profiling.${DOMAIN} → profiling-proxy:80 → flamenco-manager:8082

All services automatically get HTTPS certificates via Let's Encrypt when using real domains.

Multi-Platform Variables

Configure paths for different operating systems:

# Blender paths
BLENDER_LINUX=/usr/local/blender/blender
BLENDER_WINDOWS=C:\Program Files\Blender Foundation\Blender\blender.exe
BLENDER_DARWIN=/Applications/Blender.app/Contents/MacOS/Blender

# FFmpeg paths
FFMPEG_LINUX=/usr/bin/ffmpeg
FFMPEG_WINDOWS=C:\ffmpeg\bin\ffmpeg.exe
FFMPEG_DARWIN=/usr/local/bin/ffmpeg

Development Workflow

Core Development

# Start core services (Manager + Worker)
docker compose -f compose.dev.yml up -d

# View logs
docker compose -f compose.dev.yml logs -f

# Restart services
docker compose -f compose.dev.yml restart

# Stop services  
docker compose -f compose.dev.yml down

Frontend Development

Start the Vue.js development server with hot-reloading:

# Start development tools
docker compose -f compose.dev.yml --profile dev-tools up -d webapp-dev

# The webapp will be available at http://localhost:8081
# Changes to web/app/* files will trigger hot-reload

API Development

1. Edit OpenAPI spec:

   # Edit pkg/api/flamenco-openapi.yaml
   

2. Regenerate code:

   docker compose -f compose.dev.yml exec flamenco-manager ./mage generate
   

3. Restart to apply changes:

   docker compose -f compose.dev.yml restart flamenco-manager
   

Database Development

# Access database tools
docker compose -f compose.dev.yml --profile dev-tools run dev-tools bash

# Inside the container:
goose -dir ./internal/manager/persistence/migrations/ sqlite3 /data/flamenco-manager.sqlite status
goose -dir ./internal/manager/persistence/migrations/ sqlite3 /data/flamenco-manager.sqlite up
go run ./cmd/sqlc-export-schema
sqlc generate

Testing

# Run tests in container
docker compose -f compose.dev.yml exec flamenco-manager ./mage check

# Or build test stage
docker build -f Dockerfile.dev --target test .

Volumes and Data Persistence

Persistent Volumes

• flamenco-data: Manager database and configuration
• flamenco-shared: Shared storage for render files
• worker-data: Worker database and local data

Development Cache Volumes

• go-mod-cache: Go module cache
• yarn-cache: Node.js/Yarn cache

Data Access

# Backup data
docker run --rm -v flamenco-dev-data:/data -v $(pwd):/backup alpine tar czf /backup/flamenco-data-backup.tar.gz -C /data .

# Restore data
docker run --rm -v flamenco-dev-data:/data -v $(pwd):/backup alpine tar xzf /backup/flamenco-data-backup.tar.gz -C /data

Performance Profiling

Enable profiling with ENABLE_PPROF=true in .env:

# Start profiling session
go tool pprof -http :8083 http://localhost:8082/debug/pprof/profile?seconds=60

# View in browser at http://localhost:8083

Troubleshooting

Common Issues

1. Port conflicts:

   # Check what's using ports
   sudo lsof -i :8080
   
   # Change ports in .env file
   MANAGER_PORT=8090
   

2. Permission issues:

   # Fix volume permissions
   docker compose -f compose.dev.yml down
   docker volume rm flamenco-dev-data flamenco-dev-shared-storage
   ./scripts/dev-setup.sh
   

3. Build cache issues:

   # Clean build cache
   docker builder prune -a
   docker compose -f compose.dev.yml build --no-cache
   

4. Container won't start:

   # Check logs
   docker compose -f compose.dev.yml logs flamenco-manager
   
   # Check container status
   docker compose -f compose.dev.yml ps
   

Logs and Debugging

# View all logs
docker compose -f compose.dev.yml logs

# Follow specific service logs
docker compose -f compose.dev.yml logs -f flamenco-manager

# Execute commands in container
docker compose -f compose.dev.yml exec flamenco-manager bash

# Debug networking
docker network inspect flamenco-dev-network

Production Build

Build production-ready images:

# Build production image
docker build -f Dockerfile.dev --target production -t flamenco:latest .

# Run production container
docker run -d \
  --name flamenco-manager-prod \
  -p 8080:8080 \
  -v flamenco-prod-data:/data \
  -v flamenco-prod-storage:/shared-storage \
  flamenco:latest

Scripts Reference

Development Setup Script

./scripts/dev-setup.sh [command]

Commands:

• setup (default): Full environment setup
• dev-tools: Start development tools
• status: Show service status
• logs: Show recent logs
• restart: Restart all services
• clean: Clean up environment and volumes

Integration with Host Development

VS Code Integration

Add to .vscode/settings.json:

{
  "go.toolsEnvVars": {
    "GOFLAGS": "-tags=containers"
  },
  "docker.defaultRegistryPath": "flamenco"
}

Git Integration

The setup preserves your git configuration and allows normal development workflow:

# Normal git operations work
git add .
git commit -m "Development changes"
git push

# Code generation is available
./scripts/dev-setup.sh
docker-compose -f docker-compose.dev.yml exec flamenco-manager ./mage generate

This Docker development environment provides a complete, reproducible setup for Flamenco development with hot-reloading, debugging support, and production parity.