video-processor/docker-compose.integration.yml
Ryan Malloy 1a7d48f171 Add comprehensive end-to-end Docker integration tests
Implements complete integration test suite that validates the entire
video processing system in a containerized environment.

## Core Features
- **Video Processing Pipeline Tests**: Complete E2E validation including
  encoding, thumbnails, sprites, and metadata extraction
- **Procrastinate Worker Integration**: Async job processing, queue
  management, and error handling with version compatibility
- **Database Migration Testing**: Schema creation, version compatibility,
  and production-like migration workflows
- **Docker Orchestration**: Dedicated test environment with PostgreSQL,
  workers, and proper service dependencies

## Test Infrastructure
- **43 integration test cases** covering all major functionality
- **Containerized test environment** isolated from development
- **Automated CI/CD pipeline** with GitHub Actions
- **Performance benchmarking** and resource usage validation
- **Comprehensive error scenarios** and edge case handling

## Developer Tools
- `./scripts/run-integration-tests.sh` - Full-featured test runner
- `Makefile` - Simplified commands for common tasks
- `docker-compose.integration.yml` - Dedicated test environment
- GitHub Actions workflow with test matrix and artifact upload

## Test Coverage
- Multi-format video encoding (MP4, WebM, OGV)
- Quality preset validation (low, medium, high, ultra)
- Async job submission and processing
- Worker version compatibility (Procrastinate 2.x/3.x)
- Database schema migrations and rollbacks
- Concurrent processing scenarios
- Performance benchmarks and timeouts

Files Added:
- tests/integration/ - Complete test suite with fixtures
- docker-compose.integration.yml - Test environment configuration
- scripts/run-integration-tests.sh - Test runner with advanced options
- .github/workflows/integration-tests.yml - CI/CD pipeline
- Makefile - Development workflow automation
- Enhanced pyproject.toml with integration test dependencies

Usage:
```bash
make test-integration                    # Run all integration tests
./scripts/run-integration-tests.sh -v   # Verbose output
./scripts/run-integration-tests.sh -k   # Keep containers for debugging
make docker-test                        # Clean Docker test run
```
2025-09-05 11:24:08 -06:00

102 lines
3.3 KiB
YAML

# Docker Compose configuration for integration testing
# Separate from main docker-compose.yml to avoid conflicts during testing
version: '3.8'
services:
# PostgreSQL for integration tests
postgres-integration:
image: postgres:15-alpine
environment:
POSTGRES_DB: video_processor_integration_test
POSTGRES_USER: video_user
POSTGRES_PASSWORD: video_password
POSTGRES_HOST_AUTH_METHOD: trust
ports:
- "5433:5432" # Different port to avoid conflicts
healthcheck:
test: ["CMD-SHELL", "pg_isready -U video_user -d video_processor_integration_test"]
interval: 5s
timeout: 5s
retries: 10
networks:
- integration_net
tmpfs:
- /var/lib/postgresql/data # Use tmpfs for faster test database
# Migration service for integration tests
migrate-integration:
build:
context: .
dockerfile: Dockerfile
target: migration
environment:
- PROCRASTINATE_DATABASE_URL=postgresql://video_user:video_password@postgres-integration:5432/video_processor_integration_test
depends_on:
postgres-integration:
condition: service_healthy
networks:
- integration_net
command: ["python", "-c", "
import asyncio;
from video_processor.tasks.migration import migrate_database;
asyncio.run(migrate_database('postgresql://video_user:video_password@postgres-integration:5432/video_processor_integration_test'))
"]
# Background worker for integration tests
worker-integration:
build:
context: .
dockerfile: Dockerfile
target: worker
environment:
- PROCRASTINATE_DATABASE_URL=postgresql://video_user:video_password@postgres-integration:5432/video_processor_integration_test
- WORKER_CONCURRENCY=2 # Reduced for testing
- WORKER_TIMEOUT=60 # Faster timeout for tests
depends_on:
postgres-integration:
condition: service_healthy
migrate-integration:
condition: service_completed_successfully
networks:
- integration_net
volumes:
- integration_uploads:/app/uploads
- integration_outputs:/app/outputs
command: ["python", "-m", "video_processor.tasks.worker_compatibility", "worker"]
# Integration test runner
integration-tests:
build:
context: .
dockerfile: Dockerfile
target: development
environment:
- DATABASE_URL=postgresql://video_user:video_password@postgres-integration:5432/video_processor_integration_test
- PROCRASTINATE_DATABASE_URL=postgresql://video_user:video_password@postgres-integration:5432/video_processor_integration_test
- PYTEST_ARGS=${PYTEST_ARGS:--v --tb=short}
volumes:
- .:/app
- integration_uploads:/app/uploads
- integration_outputs:/app/outputs
- /var/run/docker.sock:/var/run/docker.sock # Access to Docker for container management
depends_on:
postgres-integration:
condition: service_healthy
migrate-integration:
condition: service_completed_successfully
worker-integration:
condition: service_started
networks:
- integration_net
command: ["uv", "run", "pytest", "tests/integration/", "-v", "--tb=short", "--durations=10"]
volumes:
integration_uploads:
driver: local
integration_outputs:
driver: local
networks:
integration_net:
driver: bridge