MCP/mcptesta
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
mcptesta/examples/templates/integration_template.yaml
Ryan Malloy bea4a2e5d3 Initial release: MCPTesta v1.0.0 🧪✨ 
Community-driven testing excellence for the MCP ecosystem

MCPTesta is a comprehensive testing framework for FastMCP servers that brings
scientific rigor and enterprise-grade capabilities to MCP protocol testing.

🎯 Core Features:
• Comprehensive FastMCP server testing with advanced protocol support
• Parallel execution with intelligent dependency resolution
• Flexible CLI and YAML configuration system
• Rich reporting: console, HTML, JSON, and JUnit formats
• Advanced MCP protocol features: notifications, cancellation, progress tracking
• Production-ready Docker environment with caddy-docker-proxy integration

🧪 Advanced Testing Capabilities:
• Multi-transport support (stdio, SSE, WebSocket)
• Authentication testing (Bearer tokens, OAuth flows)
• Stress testing and performance validation
• Memory profiling and leak detection
• CI/CD integration with comprehensive reporting

🎨 Professional Assets:
• Complete logo package with lab experiment theme
• Comprehensive documentation with Diátaxis framework
• Community-focused branding and messaging
• Multi-platform favicon and social media assets

📚 Documentation:
• Getting started tutorials and comprehensive guides
• Complete CLI and YAML reference documentation
• Architecture explanations and testing strategies
• Team collaboration and security compliance guides

🚀 Ready for:
• Community contributions and external development
• Enterprise deployment and production use
• Integration with existing FastMCP workflows
• Extension and customization for specific needs

Built with modern Python practices using uv, FastMCP, and Starlight documentation.
Designed for developers who demand scientific precision in their testing tools.

Repository: https://git.supported.systems/mcp/mcptesta
Documentation: https://mcptesta.l.supported.systems
2025-09-20 03:20:49 -06:00

611 lines
21 KiB
YAML
Raw Permalink Blame History

# MCPTesta Integration Testing Configuration Template
#
# Comprehensive integration testing template for multi-service environments.
# Tests real-world scenarios with multiple FastMCP servers, external systems,
# and complex workflow orchestration.
#
# Integration Testing Scenarios:
# - Multi-service coordination and communication
# - External system integration (databases, APIs, message queues)
# - End-to-end workflow validation
# - Cross-service transaction management
# - Service mesh and discovery integration
# - Event-driven architecture validation
# Integration testing optimized configuration
config:
parallel_workers: 8
output_directory: "./integration_test_results"
output_format: "html" # Rich visualization for complex scenarios
global_timeout: 600 # 10 minutes for complex integration scenarios
max_concurrent_operations: 25
# Integration-specific features
enable_distributed_tracing: true
enable_transaction_monitoring: true
enable_service_discovery: true
features:
test_notifications: true
test_progress: true
test_cancellation: true
test_auth: true
test_distributed_coordination: true
# Integration-friendly retry policy
retry_policy:
max_retries: 3
backoff_factor: 2.0
retry_on_errors: ["ConnectionError", "TimeoutError", "ServiceUnavailable"]
circuit_breaker:
failure_threshold: 5
recovery_timeout: 30
# Service discovery and coordination
service_discovery:
provider: "consul" # consul, etcd, kubernetes
health_check_interval: 10
service_registration: true
# Distributed tracing configuration
tracing:
enabled: true
sampler: "probabilistic"
sampling_rate: 1.0 # 100% sampling for integration tests
exporter: "jaeger"
# Multi-service environment setup
servers:
# Core business services
- name: "user_service"
command: "${USER_SERVICE_CMD:python -m user_service --port 8001}"
transport: "sse"
timeout: 30
enabled: true
env_vars:
SERVICE_NAME: "user_service"
DATABASE_URL: "${USER_DB_URL:postgresql://localhost/users}"
CACHE_URL: "${CACHE_URL:redis://localhost:6379/0}"
headers:
"Service-Version": "1.0"
"Environment": "${ENVIRONMENT:integration}"
- name: "order_service"
command: "${ORDER_SERVICE_CMD:python -m order_service --port 8002}"
transport: "sse"
timeout: 30
enabled: true
env_vars:
SERVICE_NAME: "order_service"
DATABASE_URL: "${ORDER_DB_URL:postgresql://localhost/orders}"
MESSAGE_QUEUE_URL: "${MQ_URL:amqp://localhost:5672}"
depends_on: ["user_service"]
- name: "payment_service"
command: "${PAYMENT_SERVICE_CMD:python -m payment_service --port 8003}"
transport: "sse"
timeout: 45 # Longer timeout for payment processing
enabled: true
env_vars:
SERVICE_NAME: "payment_service"
PAYMENT_GATEWAY_URL: "${PAYMENT_GATEWAY:https://api.stripe.com}"
ENCRYPTION_KEY: "${PAYMENT_ENCRYPTION_KEY}"
auth_token: "${PAYMENT_SERVICE_TOKEN}"
- name: "inventory_service"
command: "${INVENTORY_SERVICE_CMD:python -m inventory_service --port 8004}"
transport: "sse"
timeout: 30
enabled: true
env_vars:
SERVICE_NAME: "inventory_service"
DATABASE_URL: "${INVENTORY_DB_URL:postgresql://localhost/inventory}"
WAREHOUSE_API_URL: "${WAREHOUSE_API:http://localhost:9001}"
- name: "notification_service"
command: "${NOTIFICATION_SERVICE_CMD:python -m notification_service --port 8005}"
transport: "ws" # WebSocket for real-time notifications
timeout: 30
enabled: true
env_vars:
SERVICE_NAME: "notification_service"
EMAIL_PROVIDER: "${EMAIL_PROVIDER:sendgrid}"
SMS_PROVIDER: "${SMS_PROVIDER:twilio}"
# External system adapters
- name: "database_adapter"
command: "${DB_ADAPTER_CMD:python -m database_adapter --port 8006}"
transport: "stdio"
timeout: 30
enabled: true
env_vars:
SUPPORTED_DBS: "postgresql,mysql,mongodb"
CONNECTION_POOL_SIZE: "20"
- name: "message_queue_adapter"
command: "${MQ_ADAPTER_CMD:python -m mq_adapter --port 8007}"
transport: "stdio"
timeout: 30
enabled: true
env_vars:
SUPPORTED_QUEUES: "rabbitmq,kafka,sqs"
BATCH_SIZE: "100"
# Comprehensive integration test suites
test_suites:
- name: "Service Connectivity Matrix"
description: "Validate connectivity between all services"
enabled: true
tags: ["connectivity", "matrix", "health"]
parallel: false # Sequential for proper dependency validation
timeout: 180
setup:
wait_for_service_startup: 30
validate_service_registration: true
establish_baseline_health: true
tests:
- name: "service_discovery_validation"
description: "Validate all services are discoverable"
test_type: "tool_call"
target: "discover_services"
parameters:
expected_services: ["user", "order", "payment", "inventory", "notification"]
health_check: true
timeout: 30
tags: ["discovery", "health"]
- name: "inter_service_communication"
description: "Test communication between all service pairs"
test_type: "tool_call"
target: "test_service_matrix"
parameters:
services: ["user_service", "order_service", "payment_service"]
test_type: "ping"
timeout: 60
tags: ["communication", "matrix"]
depends_on: ["service_discovery_validation"]
- name: "service_dependency_validation"
description: "Validate service dependency chains"
test_type: "tool_call"
target: "validate_dependencies"
parameters:
dependency_graph: {
"order_service": ["user_service", "inventory_service"],
"payment_service": ["order_service", "user_service"],
"notification_service": ["order_service", "payment_service"]
}
timeout: 45
tags: ["dependencies", "validation"]
- name: "End-to-End Business Workflows"
description: "Complete business workflow integration testing"
enabled: true
tags: ["e2e", "workflows", "business"]
parallel: false # Sequential for workflow integrity
timeout: 400
tests:
- name: "user_registration_workflow"
description: "Complete user registration process"
test_type: "tool_call"
target: "user_registration"
parameters:
user_data: {
"email": "integration.test@example.com",
"name": "Integration Test User",
"phone": "+1234567890"
}
send_welcome_email: true
create_profile: true
enable_progress: true
timeout: 60
tags: ["user", "registration"]
- name: "order_placement_workflow"
description: "Complete order placement and processing"
test_type: "tool_call"
target: "place_order"
parameters:
user_id: "${USER_ID_FROM_REGISTRATION}"
items: [
{"product_id": "PROD_001", "quantity": 2},
{"product_id": "PROD_002", "quantity": 1}
]
payment_method: "credit_card"
shipping_address: {
"street": "123 Test Street",
"city": "Test City",
"zip": "12345"
}
enable_progress: true
timeout: 120
tags: ["order", "placement"]
depends_on: ["user_registration_workflow"]
- name: "payment_processing_workflow"
description: "Payment processing and validation"
test_type: "tool_call"
target: "process_payment"
parameters:
order_id: "${ORDER_ID_FROM_PLACEMENT}"
payment_details: {
"method": "credit_card",
"amount": "${ORDER_TOTAL}",
"currency": "USD"
}
fraud_check: true
enable_progress: true
timeout: 90
tags: ["payment", "processing"]
depends_on: ["order_placement_workflow"]
- name: "inventory_update_workflow"
description: "Inventory updates and stock management"
test_type: "tool_call"
target: "update_inventory"
parameters:
order_id: "${ORDER_ID_FROM_PLACEMENT}"
reservation_type: "confirmed"
update_warehouse: true
timeout: 45
tags: ["inventory", "update"]
depends_on: ["payment_processing_workflow"]
- name: "notification_workflow"
description: "Multi-channel notification delivery"
test_type: "tool_call"
target: "send_notifications"
parameters:
user_id: "${USER_ID_FROM_REGISTRATION}"
order_id: "${ORDER_ID_FROM_PLACEMENT}"
notification_types: ["email", "sms", "push"]
templates: ["order_confirmation", "payment_receipt"]
timeout: 60
tags: ["notifications", "delivery"]
depends_on: ["inventory_update_workflow"]
- name: "Cross-Service Transaction Testing"
description: "Distributed transaction management and consistency"
enabled: true
tags: ["transactions", "consistency", "distributed"]
parallel: false
timeout: 300
tests:
- name: "two_phase_commit_test"
description: "Test two-phase commit across services"
test_type: "tool_call"
target: "distributed_transaction"
parameters:
transaction_type: "2pc"
participants: ["user_service", "order_service", "payment_service"]
operations: [
{"service": "user_service", "action": "reserve_credit"},
{"service": "order_service", "action": "create_order"},
{"service": "payment_service", "action": "charge_card"}
]
enable_progress: true
timeout: 120
tags: ["2pc", "distributed"]
- name: "saga_pattern_test"
description: "Test saga pattern for long-running transactions"
test_type: "tool_call"
target: "saga_coordinator"
parameters:
saga_definition: {
"steps": [
{"service": "inventory", "action": "reserve", "compensate": "release"},
{"service": "payment", "action": "charge", "compensate": "refund"},
{"service": "shipping", "action": "create_label", "compensate": "cancel"}
]
}
compensation_strategy: "reverse_order"
enable_progress: true
timeout: 180
tags: ["saga", "compensation"]
- name: "eventual_consistency_test"
description: "Test eventual consistency patterns"
test_type: "tool_call"
target: "consistency_validator"
parameters:
consistency_model: "eventual"
propagation_timeout: 30
validation_points: ["immediate", "5s", "15s", "30s"]
timeout: 60
tags: ["consistency", "eventual"]
- name: "Event-Driven Architecture Testing"
description: "Event sourcing and message-driven integration"
enabled: true
tags: ["events", "messaging", "async"]
parallel: true
timeout: 250
tests:
- name: "event_publication_test"
description: "Test event publication and routing"
test_type: "tool_call"
target: "event_publisher"
parameters:
events: [
{"type": "UserRegistered", "data": {"user_id": "123"}},
{"type": "OrderPlaced", "data": {"order_id": "456"}},
{"type": "PaymentProcessed", "data": {"payment_id": "789"}}
]
routing_keys: ["user.registered", "order.placed", "payment.processed"]
timeout: 30
tags: ["events", "publication"]
- name: "event_subscription_test"
description: "Test event subscription and handling"
test_type: "notification"
target: "event_subscription"
parameters:
event_types: ["UserRegistered", "OrderPlaced", "PaymentProcessed"]
subscription_durability: "persistent"
timeout: 60
tags: ["events", "subscription"]
- name: "event_sourcing_replay_test"
description: "Test event sourcing and replay capabilities"
test_type: "tool_call"
target: "event_sourcing"
parameters:
aggregate_type: "Order"
event_sequence: [
{"type": "OrderCreated", "timestamp": "2024-01-01T00:00:00Z"},
{"type": "ItemAdded", "timestamp": "2024-01-01T00:01:00Z"},
{"type": "PaymentProcessed", "timestamp": "2024-01-01T00:02:00Z"}
]
replay_validation: true
timeout: 45
tags: ["sourcing", "replay"]
- name: "message_ordering_test"
description: "Test message ordering guarantees"
test_type: "tool_call"
target: "message_order_validator"
parameters:
message_count: 1000
ordering_key: "user_id"
validation_type: "strict"
timeout: 90
tags: ["messaging", "ordering"]
- name: "External System Integration"
description: "Integration with external systems and third-party services"
enabled: true
tags: ["external", "third_party", "integration"]
parallel: true
timeout: 300
tests:
- name: "database_integration_test"
description: "Multi-database integration testing"
test_type: "tool_call"
target: "database_coordinator"
parameters:
databases: [
{"type": "postgresql", "name": "primary"},
{"type": "redis", "name": "cache"},
{"type": "mongodb", "name": "analytics"}
]
operations: ["read", "write", "transaction", "backup"]
timeout: 60
tags: ["database", "multi_db"]
- name: "payment_gateway_integration"
description: "Payment gateway integration testing"
test_type: "tool_call"
target: "payment_gateway"
parameters:
gateway: "stripe"
test_scenarios: [
{"type": "successful_payment", "amount": 100},
{"type": "declined_card", "amount": 200},
{"type": "expired_card", "amount": 150}
]
webhook_validation: true
timeout: 90
tags: ["payment", "gateway"]
- name: "email_service_integration"
description: "Email service provider integration"
test_type: "tool_call"
target: "email_service"
parameters:
provider: "sendgrid"
email_types: ["transactional", "marketing", "notification"]
template_validation: true
delivery_tracking: true
timeout: 45
tags: ["email", "service"]
- name: "monitoring_system_integration"
description: "Monitoring and observability system integration"
test_type: "tool_call"
target: "monitoring_integration"
parameters:
systems: ["prometheus", "grafana", "jaeger", "elasticsearch"]
metrics_validation: true
alerting_test: true
timeout: 60
tags: ["monitoring", "observability"]
- name: "Service Mesh and Discovery"
description: "Service mesh integration and service discovery testing"
enabled: true
tags: ["service_mesh", "discovery", "networking"]
parallel: true
timeout: 200
tests:
- name: "service_mesh_routing"
description: "Test service mesh routing and load balancing"
test_type: "tool_call"
target: "mesh_router"
parameters:
mesh_provider: "istio"
routing_rules: [
{"service": "user_service", "weight": 80, "version": "v1"},
{"service": "user_service", "weight": 20, "version": "v2"}
]
load_balancing: "round_robin"
timeout: 60
tags: ["mesh", "routing"]
- name: "circuit_breaker_integration"
description: "Test circuit breaker patterns in service mesh"
test_type: "tool_call"
target: "circuit_breaker"
parameters:
failure_threshold: 5
timeout: 30
half_open_requests: 3
target_service: "payment_service"
timeout: 90
tags: ["circuit_breaker", "resilience"]
- name: "service_discovery_failover"
description: "Test service discovery and failover scenarios"
test_type: "tool_call"
target: "discovery_failover"
parameters:
primary_instance: "user_service_1"
backup_instances: ["user_service_2", "user_service_3"]
failover_time: 10
timeout: 60
tags: ["discovery", "failover"]
- name: "Performance and Scalability Integration"
description: "Integration performance testing under realistic load"
enabled: true
tags: ["performance", "scalability", "load"]
parallel: true
timeout: 400
tests:
- name: "end_to_end_performance"
description: "End-to-end workflow performance testing"
test_type: "tool_call"
target: "e2e_performance"
parameters:
workflow: "complete_order_process"
concurrent_users: 100
test_duration: 300
sla_requirements: {
"max_response_time": 5000,
"min_throughput": 50,
"max_error_rate": 0.01
}
enable_progress: true
timeout: 360
tags: ["e2e", "performance"]
- name: "service_scaling_test"
description: "Test service auto-scaling behavior"
test_type: "tool_call"
target: "scaling_validator"
parameters:
scaling_policy: "cpu_based"
min_instances: 2
max_instances: 10
scale_up_threshold: 70
scale_down_threshold: 30
timeout: 240
tags: ["scaling", "auto_scaling"]
- name: "database_connection_pooling"
description: "Test database connection pooling under load"
test_type: "tool_call"
target: "connection_pool_test"
parameters:
pool_size: 20
concurrent_connections: 100
connection_lifecycle: "managed"
leak_detection: true
timeout: 120
tags: ["database", "pooling"]
# Integration testing variables
variables:
# Service URLs and commands
USER_SERVICE_CMD: "python -m user_service --port 8001 --env integration"
ORDER_SERVICE_CMD: "python -m order_service --port 8002 --env integration"
PAYMENT_SERVICE_CMD: "python -m payment_service --port 8003 --env integration"
INVENTORY_SERVICE_CMD: "python -m inventory_service --port 8004 --env integration"
NOTIFICATION_SERVICE_CMD: "python -m notification_service --port 8005 --env integration"
# Database connections
USER_DB_URL: "postgresql://test_user:password@localhost:5432/users_test"
ORDER_DB_URL: "postgresql://test_user:password@localhost:5432/orders_test"
INVENTORY_DB_URL: "postgresql://test_user:password@localhost:5432/inventory_test"
CACHE_URL: "redis://localhost:6379/0"
# Message queue and external services
MQ_URL: "amqp://guest:guest@localhost:5672/"
PAYMENT_GATEWAY: "https://api.sandbox.stripe.com"
EMAIL_PROVIDER: "sendgrid_test"
SMS_PROVIDER: "twilio_test"
# Authentication tokens
PAYMENT_SERVICE_TOKEN: "${PAYMENT_TOKEN}"
PAYMENT_ENCRYPTION_KEY: "${ENCRYPTION_KEY}"
# Test environment
ENVIRONMENT: "integration"
# Dynamic values from test execution
USER_ID_FROM_REGISTRATION: "dynamic"
ORDER_ID_FROM_PLACEMENT: "dynamic"
ORDER_TOTAL: "dynamic"
# Integration Testing Best Practices:
#
# 1. Service Dependency Management:
# - Use depends_on to ensure proper startup order
# - Validate service health before running tests
# - Implement proper cleanup between test runs
#
# 2. Test Data Management:
# - Use test-specific databases and clean state
# - Implement data factories for consistent test data
# - Clean up test data after each test run
#
# 3. External System Mocking:
# - Use test/sandbox environments for external services
# - Mock external dependencies when full integration isn't possible
# - Validate contract compliance with real services
#
# 4. Error Scenario Testing:
# - Test failure modes and recovery scenarios
# - Validate circuit breaker and timeout behaviors
# - Test partial failure scenarios
#
# 5. Performance Considerations:
# - Include realistic load in integration tests
# - Monitor resource usage across all services
# - Validate SLA requirements under integration load
#
# Execution Examples:
#
# Full integration suite:
# mcptesta yaml integration_config.yaml --parallel 8 --output ./integration_results
#
# Workflow-focused testing:
# mcptesta yaml integration_config.yaml --tag workflows --tag e2e
#
# Performance integration testing:
# mcptesta yaml integration_config.yaml --tag performance --enable-profiling
#
# External system integration only:
# mcptesta yaml integration_config.yaml --tag external --tag third_party
#
# Service mesh testing:
# mcptesta yaml integration_config.yaml --tag service_mesh --tag discovery
Reference in New Issue View Git Blame Copy Permalink