flamenco/web/project-website/content/development/docker-development/how-to.md

---
title: "Troubleshooting Guide"
weight: 20
description: "Practical solutions to common Docker and Flamenco development issues"
---

# Docker Development Troubleshooting Guide

This guide provides practical solutions to common problems you might encounter with the Flamenco Docker development environment. Each section addresses a specific issue with step-by-step resolution instructions.

## Build Issues

### Go Module Download Failures

**Problem**: Build fails with network errors like "RPC failed; curl 56 Recv failure: Connection reset by peer" when downloading Go modules.

**Solution**: This indicates the Go module proxy configuration is incorrect. Verify your build is using the optimized proxy configuration:

```bash
# Check current Docker build configuration
grep -A5 "GOPROXY" Dockerfile.dev

# Should show:
# ENV GOPROXY=https://proxy.golang.org,direct
# ENV GOSUMDB=sum.golang.org
```

If you see `GOPROXY=direct`, update the Dockerfile and rebuild:

```bash
make -f Makefile.docker dev-rebuild
```

**Root cause**: Direct Git access (`GOPROXY=direct`) bypasses the proxy and is unreliable for large dependency trees.

### "pip: command not found" in Alpine

**Problem**: Build fails with Python errors like `pip: command not found` or package installation issues.

**Solution**: The issue is Alpine Linux uses `pip3` instead of `pip`. Check the Dockerfile has the correct packages:

```bash
# Verify Alpine Python packages are installed
grep -A10 "apk add" Dockerfile.dev | grep python

# Should include:
# python3
# python3-dev  
# py3-pip
```

If missing, the Docker image needs the correct packages. Also ensure pip commands use `pip3`:

```dockerfile
RUN pip3 install --no-cache-dir uv
```

**Root cause**: Alpine Linux doesn't symlink `pip` to `pip3` by default.

### Docker Layer Cache Issues

**Problem**: Builds are slower than expected, not utilizing cached layers effectively.

**Solution**: Check if you're accidentally invalidating the cache:

```bash
# Build with cache output to see what's being cached
docker compose --progress plain -f compose.dev.yml build

# Look for "CACHED" vs "RUN" in the output
```

Common cache-busting issues:
- Copying source code before dependencies
- Using `--no-cache` unnecessarily
- Timestamps in ADD/COPY operations

Fix by ensuring dependency installation happens before source code copy:

```dockerfile
# Copy dependency files first (cached layer)
COPY go.mod go.sum ./
COPY web/app/package.json web/app/yarn.lock ./web/app/

# Install dependencies (cached layer)
RUN go mod download
RUN yarn install

# Copy source code last (changes frequently)
COPY . .
```

### Binary Mount Override Problems

**Problem**: Built binaries inside the container don't work, giving "not found" or permission errors.

**Solution**: This occurs when Docker bind mounts override the `/app` directory. The optimized configuration places binaries in `/usr/local/bin` to avoid this:

```dockerfile
# Copy binaries to system location to avoid mount override
RUN cp flamenco-manager /usr/local/bin/ && cp flamenco-worker /usr/local/bin/
```

If you're still having issues, check your volume mounts don't override binary locations:

```yaml
# In compose.dev.yml - avoid mounting over binaries
volumes:
  - .:/app
  - /app/node_modules  # Prevent override
```

## Runtime Issues

### Services Won't Start

**Problem**: `make -f Makefile.docker dev-start` completes but services don't respond.

**Diagnosis steps**:
```bash
# Check service status
make -f Makefile.docker status

# Check logs for errors
make -f Makefile.docker logs

# Check specific service logs
make -f Makefile.docker logs-manager
```

**Common solutions**:

1. **Port conflicts**: Another service is using port 9000
   ```bash
   # Check what's using the port
   lsof -i :9000
   
   # Change port in .env if needed
   MANAGER_PORT=9001
   ```

2. **Database initialization**: Manager can't access database
   ```bash
   # Check database volume exists
   docker volume ls | grep flamenco-dev-data
   
   # If missing, recreate with setup
   make -f Makefile.docker dev-clean
   make -f Makefile.docker dev-setup
   ```

3. **Network issues**: Services can't communicate
   ```bash
   # Recreate networks
   make -f Makefile.docker dev-stop
   docker network rm flamenco-dev-network 2>/dev/null
   make -f Makefile.docker network-setup
   make -f Makefile.docker dev-start
   ```

### Worker Won't Connect

**Problem**: Worker service starts but doesn't appear in the Manager's worker list.

**Diagnosis**:
```bash
# Check worker logs for connection errors
make -f Makefile.docker logs-worker
```

**Solutions**:

1. **Manager not ready**: Worker starts before Manager is fully initialized
   ```bash
   # Wait for Manager health check, then restart worker
   make -f Makefile.docker status
   docker restart flamenco-dev-worker
   ```

2. **Network configuration**: Worker can't reach Manager
   ```bash
   # Verify worker can reach Manager
   docker exec flamenco-dev-worker curl -s http://flamenco-manager:8080/api/v3/version
   ```

3. **Configuration mismatch**: Manager URL is incorrect
   ```bash
   # Check worker environment
   docker exec flamenco-dev-worker env | grep MANAGER_URL
   
   # Should be: MANAGER_URL=http://flamenco-manager:8080
   ```

### Web Interface Not Accessible

**Problem**: Cannot access Flamenco Manager at http://localhost:9000.

**Diagnosis**:
```bash
# Test direct connection
curl -v http://localhost:9000/api/v3/version

# Check port binding
docker port flamenco-dev-manager
```

**Solutions**:

1. **Service not running**: Manager container stopped
   ```bash
   make -f Makefile.docker dev-start
   ```

2. **Port mapping**: Wrong port configuration
   ```bash
   # Check .env file
   grep MANAGER_PORT .env
   
   # Should match docker port mapping
   grep "MANAGER_PORT" compose.dev.yml
   ```

3. **Firewall/proxy**: Local firewall blocking connection
   ```bash
   # Test from inside container
   docker exec flamenco-dev-manager curl -s localhost:8080/api/v3/version
   ```

## Performance Issues

### Slow Build Times

**Problem**: Docker builds take much longer than the expected 26 minutes.

**Investigation**:
```bash
# Build with timing information
time docker compose --progress plain -f compose.dev.yml build
```

**Common causes and solutions**:

1. **No proxy caching**: Go modules downloading directly
   - Verify `GOPROXY=https://proxy.golang.org,direct` in Dockerfile
   - Check network connectivity to proxy.golang.org

2. **Resource constraints**: Insufficient CPU/memory allocated to Docker
   ```bash
   # Check Docker resource settings
   docker system info | grep -A5 "CPUs\|Memory"
   
   # Increase Docker memory allocation if under 4GB
   ```

3. **Dependency changes**: Frequent cache invalidation
   - Avoid changing `go.mod`, `package.json`, or `pyproject.toml` frequently
   - Use separate commits for dependency vs code changes

### High Memory Usage

**Problem**: Docker containers consuming excessive memory.

**Diagnosis**:
```bash
# Check container resource usage
docker stats

# Check system memory
free -h
```

**Solutions**:

1. **Set memory limits**: Add limits to compose.dev.yml
   ```yaml
   services:
     flamenco-manager:
       deploy:
         resources:
           limits:
             memory: 1G
   ```

2. **Clean up unused resources**:
   ```bash
   # Remove unused containers and images
   make -f Makefile.docker clean-all
   
   # Clean Docker system
   docker system prune -f
   ```

## Development Workflow Issues

### Hot Reload Not Working

**Problem**: Code changes don't trigger rebuilds or container updates.

**Check bind mounts**:
```bash
# Verify source code is mounted
docker exec flamenco-dev-manager ls -la /app

# Should show your local files with recent timestamps
```

**Solutions**:

1. **Restart development services**:
   ```bash
   make -f Makefile.docker dev-restart
   ```

2. **Force rebuild with changes**:
   ```bash
   # Rebuild just the changed service
   docker compose -f compose.dev.yml up -d --build flamenco-manager
   ```

3. **Check file watchers**: Some editors don't trigger file system events
   ```bash
   # Test with direct file change
   touch internal/manager/main.go
   
   # Should trigger rebuild in Manager container
   ```

### Database Migration Failures

**Problem**: Database migrations fail during startup or development.

**Check migration status**:
```bash
make -f Makefile.docker db-status
```

**Solutions**:

1. **Manual migration**:
   ```bash
   # Apply pending migrations
   make -f Makefile.docker db-up
   ```

2. **Reset database**: For development, you can reset completely
   ```bash
   make -f Makefile.docker dev-stop
   docker volume rm flamenco-dev-data
   make -f Makefile.docker dev-setup
   ```

3. **Check migration files**: Ensure no corruption
   ```bash
   # List migration files
   ls -la internal/manager/persistence/migrations/
   ```

## Environment Configuration Issues

### Environment Variables Not Loading

**Problem**: Services start with default values instead of custom .env configuration.

**Check .env loading**:
```bash
# Verify .env file exists and is readable
cat .env

# Check if Docker Compose sees the variables
make -f Makefile.docker config | grep -A5 "environment:"
```

**Solutions**:

1. **File location**: .env must be in the same directory as compose.dev.yml
   ```bash
   ls -la .env compose.dev.yml
   # Both should be present
   ```

2. **Syntax errors**: Invalid .env format
   ```bash
   # Check for syntax issues
   grep -n "=" .env | grep -v "^[A-Z_]*="
   ```

3. **Service restart**: Changes require container restart
   ```bash
   make -f Makefile.docker dev-restart
   ```

### Permission Denied Errors

**Problem**: Containers can't write to mounted volumes or shared storage.

**Diagnosis**:
```bash
# Check volume permissions
docker exec flamenco-dev-manager ls -la /shared-storage
docker exec flamenco-dev-manager ls -la /data
```

**Solutions**:

1. **Shared storage initialization**:
   ```bash
   # Reinitialize with proper permissions
   docker compose -f compose.dev.yml up shared-storage-setup
   ```

2. **Host permission issues**: If using bind mounts
   ```bash
   # Fix host directory permissions
   sudo chown -R $USER:$USER ./flamenco-data
   chmod -R 755 ./flamenco-data
   ```

## Getting Additional Help

If these solutions don't resolve your issue:

1. **Check the logs**: Always start with `make -f Makefile.docker logs`
2. **Verify versions**: Ensure you're using supported Docker and Compose versions
3. **Clean slate**: Try `make -f Makefile.docker dev-clean` and start over
4. **Community support**: Ask on [Flamenco Chat](https://blender.chat/channel/flamenco)
5. **Report bugs**: Create an issue at [Flamenco Issues](https://projects.blender.org/studio/flamenco/issues)

When seeking help, include:
- Your operating system and Docker version
- Complete error messages from logs
- The commands you ran leading to the issue
- Your .env file (without sensitive information)