mcp-vultr/TESTING.md

Comprehensive Test Suite Documentation

This document describes the complete test suite for the Vultr DNS MCP package, following FastMCP testing best practices.

🧪 Test Structure Overview

The test suite is organized following modern Python testing practices and FastMCP patterns:

tests/
├── conftest.py                    # Test configuration and fixtures
├── test_mcp_server.py            # MCP server functionality tests
├── test_client.py                # VultrDNSClient tests  
├── test_cli.py                   # CLI interface tests
├── test_vultr_server.py          # Core VultrDNSServer tests
└── test_package_validation.py    # Package integrity tests

🎯 Testing Patterns Used

FastMCP In-Memory Testing Pattern

Following the official FastMCP testing documentation, we use the in-memory testing pattern:

@pytest.mark.asyncio
async def test_mcp_tool(mcp_server):
    async with Client(mcp_server) as client:
        result = await client.call_tool("tool_name", {"param": "value"})
        assert result is not None

This approach:

• ✅ Tests the actual MCP server without starting a separate process
• ✅ Provides fast, reliable test execution
• ✅ Allows testing of all MCP functionality in isolation
• ✅ Enables comprehensive error scenario testing

📋 Test Categories

Unit Tests (@pytest.mark.unit)

• Test individual components in isolation
• Mock external dependencies (Vultr API)
• Fast execution, no network calls
• High code coverage focus

Example files:

• Core VultrDNSServer functionality
• Client convenience methods
• Validation logic
• CLI command parsing

Integration Tests (@pytest.mark.integration)

• Test component interactions
• End-to-end workflows
• Multiple components working together
• Realistic usage scenarios

Example scenarios:

• Complete domain management workflow
• Record creation → validation → analysis flow
• CLI command chains
• Setup utility functions

MCP Tests (@pytest.mark.mcp)

• Specific to Model Context Protocol functionality
• Test MCP tools and resources
• Client-server communication
• Resource discovery

Coverage:

• All MCP tools (12 tools total)
• Resource endpoints (domains, capabilities, records)
• Error handling in MCP context
• Natural language parameter handling

Slow Tests (@pytest.mark.slow)

• Tests that take longer to execute
• Network timeout simulations
• Large data set processing
• Performance edge cases

🛠️ Test Fixtures and Mocks

Core Fixtures (conftest.py)

@pytest.fixture
def mcp_server(mock_api_key):
    """Create a FastMCP server instance for testing."""
    return create_mcp_server(mock_api_key)

@pytest.fixture
def mock_vultr_client():
    """Create a mock VultrDNSServer with realistic responses."""
    # Configured with comprehensive mock data

Mock Strategy

• VultrDNSServer: Mocked to avoid real API calls
• FastMCP Client: Real instance for authentic MCP testing
• CLI Commands: Mocked underlying clients
• HTTP Responses: Realistic Vultr API response patterns

🚀 Running Tests

Using uv (recommended):

# All tests
uv run pytest

# Specific categories
uv run pytest -m unit
uv run pytest -m integration  
uv run pytest -m mcp
uv run pytest -m "not slow"

# With coverage
uv run pytest --cov=vultr_dns_mcp --cov-report=html

Using the test runner:

# Comprehensive test runner (uv)
uv run python run_tests.py

# Specific test types
uv run python run_tests.py --type unit --verbose
uv run python run_tests.py --type mcp --coverage
uv run python run_tests.py --fast  # Skip slow tests

# Full validation
uv run python run_tests.py --all-checks

Traditional approach (fallback):

# All tests
pytest

# Test runner
python run_tests.py --all-checks

📊 Coverage Goals

• Overall Coverage: 80%+ (enforced by pytest)
• Critical Paths: 95%+ (MCP tools, API client)
• Error Handling: 100% (exception scenarios)
• CLI Commands: 90%+ (user-facing functionality)

Coverage Reports

• Terminal: Summary with missing lines
• HTML: Detailed interactive report (htmlcov/)
• XML: For CI/CD integration

🔧 Test Configuration

pytest.ini (in pyproject.toml)

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = [
    "--strict-markers",
    "--verbose", 
    "--cov=vultr_dns_mcp",
    "--cov-fail-under=80"
]
asyncio_mode = "auto"
markers = [
    "unit: Unit tests that test individual components",
    "integration: Integration tests that test interactions", 
    "mcp: Tests specifically for MCP server functionality",
    "slow: Tests that take a long time to run"
]

🎭 Mock Data Patterns

Realistic Test Data

# Domain data matches Vultr API response format
sample_domain = {
    "domain": "example.com",
    "date_created": "2024-01-01T00:00:00Z", 
    "dns_sec": "disabled"
}

# Record data includes all typical fields
sample_record = {
    "id": "record-123",
    "type": "A",
    "name": "www", 
    "data": "192.168.1.100",
    "ttl": 300,
    "priority": None
}

Error Simulation

• API error responses (400, 401, 500)
• Network timeouts and connection failures
• Rate limiting scenarios
• Invalid parameter handling

🏗️ CI/CD Integration

GitHub Actions Workflow

• Multi-Python Testing: 3.8, 3.9, 3.10, 3.11, 3.12
• Test Categories: Unit → Integration → MCP
• Code Quality: Black, isort, flake8, mypy
• Security: Safety, Bandit scans
• Package Building: Wheel creation and validation
• Installation Testing: Install from wheel and test

Quality Gates

1. ✅ All tests pass on all Python versions
2. ✅ Code coverage meets 80% threshold
3. ✅ Code quality checks pass (formatting, linting, types)
4. ✅ Security scans show no issues
5. ✅ Package builds and installs correctly
6. ✅ CLI tools work after installation

🧩 Test Design Principles

Isolation

• Each test is independent and can run alone
• No shared state between tests
• Clean fixtures for each test

Realism

• Mock data matches real API responses
• Error scenarios reflect actual API behavior
• Test data covers edge cases and common patterns

Maintainability

• Clear test names describing what's being tested
• Logical test organization by functionality
• Comprehensive fixtures reducing code duplication
• Good documentation of test purpose

Speed

• Fast unit tests for quick feedback
• Slower integration tests for comprehensive validation
• Parallel execution support
• Efficient mocking to avoid network calls

📈 Testing Metrics

Current Test Count

• Unit Tests: ~40 tests
• Integration Tests: ~15 tests
• MCP Tests: ~20 tests
• CLI Tests: ~25 tests
• Total: ~100 comprehensive tests

Coverage Breakdown

• Server Module: 95%+
• Client Module: 90%+
• CLI Module: 85%+
• MCP Tools: 100%
• Error Handling: 95%+

🔮 Future Enhancements

Planned Additions

• Property-based testing with Hypothesis
• Load testing for MCP server performance
• End-to-end tests with real Vultr sandbox
• Documentation tests with doctest
• Mutation testing for test quality validation

Test Infrastructure

• Test data factories for complex scenarios
• Custom pytest plugins for MCP-specific testing
• Performance benchmarking integration
• Visual regression testing for CLI output

---

This comprehensive test suite ensures the Vultr DNS MCP package is reliable, maintainable, and ready for production use while following FastMCP best practices! 🎉