Ryan Malloy
343f989714
MAJOR ENHANCEMENTS: • Professional documentation structure in docs/ with symlinked examples • Comprehensive test organization under tests/ directory • Advanced video-themed testing framework with HTML dashboards • Enhanced Makefile with categorized test commands DOCUMENTATION RESTRUCTURE: • docs/user-guide/ - User-facing guides and features • docs/development/ - Technical documentation • docs/migration/ - Upgrade instructions • docs/reference/ - API references and roadmaps • examples/ - Practical usage examples (symlinked to docs/examples) TEST ORGANIZATION: • tests/unit/ - Unit tests with enhanced reporting • tests/integration/ - End-to-end tests • tests/docker/ - Docker integration configurations • tests/framework/ - Custom testing framework components • tests/development-archives/ - Historical test data TESTING FRAMEWORK FEATURES: • Video-themed HTML dashboards with cinema aesthetics • Quality scoring system (0-10 scale with letter grades) • Test categorization (unit, integration, 360°, AI, streaming, performance) • Parallel execution with configurable workers • Performance metrics and trend analysis • Interactive filtering and expandable test details INTEGRATION IMPROVEMENTS: • Updated docker-compose paths for new structure • Enhanced Makefile with video processing test commands • Backward compatibility with existing tests • CI/CD ready with JSON reports and exit codes • Professional quality assurance workflows TECHNICAL ACHIEVEMENTS: • 274 tests organized with smart categorization • 94.8% unit test success rate with enhanced reporting • Video processing domain-specific fixtures and assertions • Beautiful dark terminal aesthetic with video processing colors • Production-ready framework with enterprise-grade features Commands: make test-smoke, make test-unit, make test-360, make test-all Reports: Video-themed HTML dashboards in test-reports/ Quality: Comprehensive scoring and performance tracking
230 lines
6.9 KiB
Markdown
230 lines
6.9 KiB
Markdown
# Integration Tests
|
|
|
|
This directory contains end-to-end integration tests that verify the complete Video Processor system in a Docker environment.
|
|
|
|
## Overview
|
|
|
|
The integration tests validate:
|
|
|
|
- **Complete video processing pipeline** - encoding, thumbnails, sprites
|
|
- **Procrastinate worker functionality** - async job processing and queue management
|
|
- **Database migration system** - schema creation and version compatibility
|
|
- **Docker containerization** - multi-service orchestration
|
|
- **Error handling and edge cases** - real-world failure scenarios
|
|
|
|
## Test Architecture
|
|
|
|
### Test Structure
|
|
|
|
```
|
|
tests/integration/
|
|
├── conftest.py # Pytest fixtures and Docker setup
|
|
├── test_video_processing_e2e.py # Video processing pipeline tests
|
|
├── test_procrastinate_worker_e2e.py # Worker and job queue tests
|
|
├── test_database_migration_e2e.py # Database migration tests
|
|
└── README.md # This file
|
|
```
|
|
|
|
### Docker Services
|
|
|
|
The tests use a dedicated Docker Compose configuration (`tests/docker/docker-compose.integration.yml`) with:
|
|
|
|
- **postgres-integration** - PostgreSQL database on port 5433
|
|
- **migrate-integration** - Runs database migrations
|
|
- **worker-integration** - Procrastinate background worker
|
|
- **integration-tests** - Test runner container
|
|
|
|
## Running Integration Tests
|
|
|
|
### Quick Start
|
|
|
|
```bash
|
|
# Run all integration tests
|
|
make test-integration
|
|
|
|
# Or use the script directly
|
|
./scripts/run-integration-tests.sh
|
|
```
|
|
|
|
### Advanced Options
|
|
|
|
```bash
|
|
# Verbose output
|
|
./scripts/run-integration-tests.sh --verbose
|
|
|
|
# Fast mode (skip slow tests)
|
|
./scripts/run-integration-tests.sh --fast
|
|
|
|
# Run specific test pattern
|
|
./scripts/run-integration-tests.sh --test-filter "test_video_processing"
|
|
|
|
# Keep containers for debugging
|
|
./scripts/run-integration-tests.sh --keep
|
|
|
|
# Clean start
|
|
./scripts/run-integration-tests.sh --clean
|
|
```
|
|
|
|
### Manual Docker Setup
|
|
|
|
```bash
|
|
# Start services manually
|
|
docker-compose -f tests/docker/docker-compose.integration.yml up -d postgres-integration
|
|
docker-compose -f tests/docker/docker-compose.integration.yml run --rm migrate-integration
|
|
docker-compose -f tests/docker/docker-compose.integration.yml up -d worker-integration
|
|
|
|
# Run tests
|
|
docker-compose -f tests/docker/docker-compose.integration.yml run --rm integration-tests
|
|
|
|
# Cleanup
|
|
docker-compose -f tests/docker/docker-compose.integration.yml down -v
|
|
```
|
|
|
|
## Test Categories
|
|
|
|
### Video Processing Tests (`test_video_processing_e2e.py`)
|
|
|
|
- **Synchronous processing** - Complete pipeline with multiple formats
|
|
- **Configuration validation** - Quality presets and output formats
|
|
- **Error handling** - Invalid inputs and edge cases
|
|
- **Performance testing** - Processing time validation
|
|
- **Concurrent processing** - Multiple simultaneous jobs
|
|
|
|
### Worker Integration Tests (`test_procrastinate_worker_e2e.py`)
|
|
|
|
- **Job submission** - Async task queuing and processing
|
|
- **Worker functionality** - Background job execution
|
|
- **Error handling** - Failed job scenarios
|
|
- **Queue management** - Job status and monitoring
|
|
- **Version compatibility** - Procrastinate 2.x/3.x support
|
|
|
|
### Database Migration Tests (`test_database_migration_e2e.py`)
|
|
|
|
- **Fresh installation** - Database schema creation
|
|
- **Migration idempotency** - Safe re-runs
|
|
- **Version compatibility** - 2.x vs 3.x migration paths
|
|
- **Production workflows** - Multi-stage migrations
|
|
- **Error scenarios** - Invalid configurations
|
|
|
|
## Test Data
|
|
|
|
Tests use FFmpeg-generated test videos:
|
|
- 10-second test video (640x480, 30fps)
|
|
- Created dynamically using `testsrc` filter
|
|
- Small size for fast processing
|
|
|
|
## Dependencies
|
|
|
|
### System Requirements
|
|
|
|
- **Docker & Docker Compose** - Container orchestration
|
|
- **FFmpeg** - Video processing (system package)
|
|
- **PostgreSQL client** - Database testing utilities
|
|
|
|
### Python Dependencies
|
|
|
|
```toml
|
|
# Added to pyproject.toml [project.optional-dependencies.dev]
|
|
"pytest-asyncio>=0.21.0" # Async test support
|
|
"docker>=6.1.0" # Docker API client
|
|
"psycopg2-binary>=2.9.0" # PostgreSQL adapter
|
|
```
|
|
|
|
## Debugging
|
|
|
|
### View Logs
|
|
|
|
```bash
|
|
# Show all service logs
|
|
docker-compose -f tests/docker/docker-compose.integration.yml logs
|
|
|
|
# Follow specific service
|
|
docker-compose -f tests/docker/docker-compose.integration.yml logs -f worker-integration
|
|
|
|
# Test logs are saved to test-reports/ directory
|
|
```
|
|
|
|
### Connect to Services
|
|
|
|
```bash
|
|
# Access test database
|
|
psql -h localhost -p 5433 -U video_user -d video_processor_integration_test
|
|
|
|
# Execute commands in containers
|
|
docker-compose -f tests/docker/docker-compose.integration.yml exec postgres-integration psql -U video_user
|
|
|
|
# Access test container
|
|
docker-compose -f tests/docker/docker-compose.integration.yml run --rm integration-tests bash
|
|
```
|
|
|
|
### Common Issues
|
|
|
|
**Port conflicts**: Integration tests use port 5433 to avoid conflicts with main PostgreSQL
|
|
|
|
**FFmpeg missing**: Install system FFmpeg package: `sudo apt install ffmpeg`
|
|
|
|
**Docker permissions**: Add user to docker group: `sudo usermod -aG docker $USER`
|
|
|
|
**Database connection failures**: Ensure PostgreSQL container is healthy before running tests
|
|
|
|
## CI/CD Integration
|
|
|
|
### GitHub Actions
|
|
|
|
The integration tests run automatically on:
|
|
- Push to main/develop branches
|
|
- Pull requests to main
|
|
- Daily scheduled runs (2 AM UTC)
|
|
|
|
See `.github/workflows/integration-tests.yml` for configuration.
|
|
|
|
### Test Matrix
|
|
|
|
Tests run with different configurations:
|
|
- Separate test suites (video, worker, database)
|
|
- Full integration suite
|
|
- Performance testing (scheduled only)
|
|
- Security scanning
|
|
|
|
## Performance Benchmarks
|
|
|
|
Expected performance for test environment:
|
|
- Video processing: < 10x realtime for test videos
|
|
- Job processing: < 60 seconds for simple tasks
|
|
- Database migration: < 30 seconds
|
|
- Full test suite: < 20 minutes
|
|
|
|
## Contributing
|
|
|
|
When adding integration tests:
|
|
|
|
1. **Use fixtures** - Leverage `conftest.py` fixtures for setup
|
|
2. **Clean state** - Use `clean_database` fixture to isolate tests
|
|
3. **Descriptive names** - Use clear test method names
|
|
4. **Proper cleanup** - Ensure resources are freed after tests
|
|
5. **Error messages** - Provide helpful assertions with context
|
|
|
|
### Test Guidelines
|
|
|
|
- Test real scenarios users will encounter
|
|
- Include both success and failure paths
|
|
- Validate outputs completely (file existence, content, metadata)
|
|
- Keep tests fast but comprehensive
|
|
- Use meaningful test data and IDs
|
|
|
|
## Troubleshooting
|
|
|
|
### Failed Tests
|
|
|
|
1. Check container logs: `./scripts/run-integration-tests.sh --verbose`
|
|
2. Verify Docker services: `docker-compose -f tests/docker/docker-compose.integration.yml ps`
|
|
3. Test database connection: `psql -h localhost -p 5433 -U video_user`
|
|
4. Check FFmpeg: `ffmpeg -version`
|
|
|
|
### Resource Issues
|
|
|
|
- **Out of disk space**: Run `docker system prune -af`
|
|
- **Memory issues**: Reduce `WORKER_CONCURRENCY` in docker-compose
|
|
- **Network conflicts**: Use `--clean` flag to reset network state
|
|
|
|
For more help, see the main project README or open an issue. |