flamenco/DOCKER_DEVELOPMENT.md

# Docker Development Environment

This document describes how to set up and use the Flamenco development environment using Docker and Docker Compose.

## Quick Start

### Prerequisites

Ensure [caddy-docker-proxy](https://github.com/lucaslorentz/caddy-docker-proxy) is running:

```bash
docker network create caddy
docker run -d \
  --name caddy-docker-proxy \
  --restart unless-stopped \
  --network caddy \
  -p 80:80 -p 443:443 \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  -v caddy_data:/data \
  lucaslorentz/caddy-docker-proxy:ci-alpine
```

### Setup

1. **Setup the environment:**
   ```bash
   ./scripts/dev-setup.sh
   ```

2. **Access Flamenco (via reverse proxy):**
   - Manager: https://manager.flamenco.l.supported.systems
   - Manager API: https://manager.flamenco.l.supported.systems/api/v3

3. **Access Flamenco (direct):**
   - Manager Web Interface: http://localhost:8080
   - Manager API: http://localhost:8080/api/v3
   - Profiling (pprof): http://localhost:8082/debug/pprof

4. **Start development tools (optional):**
   ```bash
   ./scripts/dev-setup.sh dev-tools
   ```

5. **Development URLs (via reverse proxy):**
   - Vue.js Frontend: https://flamenco.l.supported.systems
   - Documentation: https://docs.flamenco.l.supported.systems
   - Profiling: https://profiling.flamenco.l.supported.systems

6. **Development URLs (direct):**
   - Vue.js Dev Server: http://localhost:8081
   - Hugo Documentation: http://localhost:1313

## Architecture Overview

The Docker development environment consists of multiple stages and services:

### Docker Multi-Stage Build

- **base**: Common dependencies (Go, Node.js, Java, etc.)
- **deps**: Dependencies installation and caching
- **build-tools**: Flamenco build tools (Mage, generators)
- **development**: Full development environment with hot-reloading
- **builder**: Production binary building
- **production**: Minimal runtime image
- **test**: Testing environment
- **tools**: Development utilities

### Services

- **flamenco-manager**: Central coordination server
- **flamenco-worker**: Task execution daemon
- **webapp-dev**: Vue.js development server (hot-reloading)
- **docs-dev**: Hugo documentation server
- **dev-tools**: Database and development utilities
- **shared-storage-setup**: Storage initialization

## Configuration

### Environment Variables

Copy `.env.dev` to `.env` and customize:

```bash
cp .env.dev .env
```

Key configuration options:

```bash
# Project naming
COMPOSE_PROJECT_NAME=flamenco-dev

# Domain configuration for reverse proxy
DOMAIN=flamenco.l.supported.systems

# Ports (for direct access)
MANAGER_PORT=8080
WEBAPP_DEV_PORT=8081
DOCS_DEV_PORT=1313

# Development settings
LOG_LEVEL=debug
ENABLE_PPROF=true
```

### Reverse Proxy Configuration

The environment includes caddy-docker-proxy labels for automatic SSL termination and routing:

- **Manager**: `manager.${DOMAIN}` → `flamenco-manager:8080`
- **Vue.js Frontend**: `${DOMAIN}` → `webapp-dev:8081`  
- **Documentation**: `docs.${DOMAIN}` → `docs-dev:1313`
- **Profiling**: `profiling.${DOMAIN}` → `profiling-proxy:80` → `flamenco-manager:8082`

All services automatically get HTTPS certificates via Let's Encrypt when using real domains.

### Multi-Platform Variables

Configure paths for different operating systems:

```bash
# Blender paths
BLENDER_LINUX=/usr/local/blender/blender
BLENDER_WINDOWS=C:\Program Files\Blender Foundation\Blender\blender.exe
BLENDER_DARWIN=/Applications/Blender.app/Contents/MacOS/Blender

# FFmpeg paths
FFMPEG_LINUX=/usr/bin/ffmpeg
FFMPEG_WINDOWS=C:\ffmpeg\bin\ffmpeg.exe
FFMPEG_DARWIN=/usr/local/bin/ffmpeg
```

## Development Workflow

### Core Development

```bash
# Start core services (Manager + Worker)
docker compose -f compose.dev.yml up -d

# View logs
docker compose -f compose.dev.yml logs -f

# Restart services
docker compose -f compose.dev.yml restart

# Stop services  
docker compose -f compose.dev.yml down
```

### Frontend Development

Start the Vue.js development server with hot-reloading:

```bash
# Start development tools
docker compose -f compose.dev.yml --profile dev-tools up -d webapp-dev

# The webapp will be available at http://localhost:8081
# Changes to web/app/* files will trigger hot-reload
```

### API Development

1. **Edit OpenAPI spec:**
   ```bash
   # Edit pkg/api/flamenco-openapi.yaml
   ```

2. **Regenerate code:**
   ```bash
   docker compose -f compose.dev.yml exec flamenco-manager ./mage generate
   ```

3. **Restart to apply changes:**
   ```bash
   docker compose -f compose.dev.yml restart flamenco-manager
   ```

### Database Development

```bash
# Access database tools
docker compose -f compose.dev.yml --profile dev-tools run dev-tools bash

# Inside the container:
goose -dir ./internal/manager/persistence/migrations/ sqlite3 /data/flamenco-manager.sqlite status
goose -dir ./internal/manager/persistence/migrations/ sqlite3 /data/flamenco-manager.sqlite up
go run ./cmd/sqlc-export-schema
sqlc generate
```

### Testing

```bash
# Run tests in container
docker compose -f compose.dev.yml exec flamenco-manager ./mage check

# Or build test stage
docker build -f Dockerfile.dev --target test .
```

## Volumes and Data Persistence

### Persistent Volumes

- **flamenco-data**: Manager database and configuration
- **flamenco-shared**: Shared storage for render files
- **worker-data**: Worker database and local data

### Development Cache Volumes

- **go-mod-cache**: Go module cache
- **yarn-cache**: Node.js/Yarn cache

### Data Access

```bash
# Backup data
docker run --rm -v flamenco-dev-data:/data -v $(pwd):/backup alpine tar czf /backup/flamenco-data-backup.tar.gz -C /data .

# Restore data
docker run --rm -v flamenco-dev-data:/data -v $(pwd):/backup alpine tar xzf /backup/flamenco-data-backup.tar.gz -C /data
```

## Performance Profiling

Enable profiling with `ENABLE_PPROF=true` in `.env`:

```bash
# Start profiling session
go tool pprof -http :8083 http://localhost:8082/debug/pprof/profile?seconds=60

# View in browser at http://localhost:8083
```

## Troubleshooting

### Common Issues

1. **Port conflicts:**
   ```bash
   # Check what's using ports
   sudo lsof -i :8080
   
   # Change ports in .env file
   MANAGER_PORT=8090
   ```

2. **Permission issues:**
   ```bash
   # Fix volume permissions
   docker compose -f compose.dev.yml down
   docker volume rm flamenco-dev-data flamenco-dev-shared-storage
   ./scripts/dev-setup.sh
   ```

3. **Build cache issues:**
   ```bash
   # Clean build cache
   docker builder prune -a
   docker compose -f compose.dev.yml build --no-cache
   ```

4. **Container won't start:**
   ```bash
   # Check logs
   docker compose -f compose.dev.yml logs flamenco-manager
   
   # Check container status
   docker compose -f compose.dev.yml ps
   ```

### Logs and Debugging

```bash
# View all logs
docker compose -f compose.dev.yml logs

# Follow specific service logs
docker compose -f compose.dev.yml logs -f flamenco-manager

# Execute commands in container
docker compose -f compose.dev.yml exec flamenco-manager bash

# Debug networking
docker network inspect flamenco-dev-network
```

## Production Build

Build production-ready images:

```bash
# Build production image
docker build -f Dockerfile.dev --target production -t flamenco:latest .

# Run production container
docker run -d \
  --name flamenco-manager-prod \
  -p 8080:8080 \
  -v flamenco-prod-data:/data \
  -v flamenco-prod-storage:/shared-storage \
  flamenco:latest
```

## Scripts Reference

### Development Setup Script

```bash
./scripts/dev-setup.sh [command]
```

Commands:
- `setup` (default): Full environment setup
- `dev-tools`: Start development tools
- `status`: Show service status
- `logs`: Show recent logs  
- `restart`: Restart all services
- `clean`: Clean up environment and volumes

## Integration with Host Development

### VS Code Integration

Add to `.vscode/settings.json`:

```json
{
  "go.toolsEnvVars": {
    "GOFLAGS": "-tags=containers"
  },
  "docker.defaultRegistryPath": "flamenco"
}
```

### Git Integration

The setup preserves your git configuration and allows normal development workflow:

```bash
# Normal git operations work
git add .
git commit -m "Development changes"
git push

# Code generation is available
./scripts/dev-setup.sh
docker-compose -f docker-compose.dev.yml exec flamenco-manager ./mage generate
```

This Docker development environment provides a complete, reproducible setup for Flamenco development with hot-reloading, debugging support, and production parity.