mcptesta/examples/templates/README.md

MCPTesta Configuration Templates

This directory contains comprehensive configuration templates for MCPTesta, demonstrating different complexity levels and use cases.

Available Templates

📚 Basic Template (basic_template.yaml)

Perfect for beginners and quick validation

• Simple server configuration
• Basic connectivity and tool testing
• Console output with clear documentation
• Essential features only

Use when:

• Learning MCPTesta fundamentals
• Quick server validation
• Simple testing scenarios

📈 Intermediate Template (intermediate_template.yaml)

Mid-level template with advanced features

• Multiple test suites with dependencies
• Basic MCP protocol features (notifications, progress)
• HTML reporting and environment variables
• Error handling and retry policies

Use when:

• Need dependency management between tests
• Want to explore MCP protocol features
• Require organized test suites with setup/teardown

🚀 Advanced Template (advanced_template.yaml)

Full-featured template with all MCP capabilities

• Complete MCP protocol feature testing
• Multi-server coordination
• Performance testing and profiling
• Complex scenarios and error handling

Use when:

• Production-ready testing
• Need comprehensive MCP protocol coverage
• Testing complex multi-server scenarios

🎯 Expert Template (expert_template.yaml)

Maximum complexity for expert users

• Distributed system testing
• Chaos engineering patterns
• Advanced authentication schemes
• Enterprise-grade features

Use when:

• Expert-level protocol testing
• Distributed system validation
• Chaos engineering and resilience testing
• Protocol development and research

⚡ Stress Template (stress_template.yaml)

Specialized performance and load testing

• Various load patterns and stress scenarios
• Resource exhaustion testing
• Performance benchmarking
• Long-duration stability testing

Use when:

• Performance validation
• Load testing and capacity planning
• Identifying bottlenecks and limits
• Stability and endurance testing

🔗 Integration Template (integration_template.yaml)

Multi-service and external system integration

• End-to-end workflow testing
• Service mesh and discovery
• External system integration
• Transaction management

Use when:

• Testing multi-service architectures
• Validating service interactions
• External system integration
• Workflow orchestration testing

Usage Examples

Basic Usage

# Generate a basic template
mcptesta generate-config basic my_basic_config.yaml

# Generate with custom server command
mcptesta generate-config basic my_config.yaml --server-command "uvx my-server"

Advanced Usage

# Generate advanced template with custom features
mcptesta generate-config advanced advanced_config.yaml \
  --server-command "python -m my_server --production" \
  --parallel-workers 8 \
  --enable-features "notifications,progress,cancellation"

# Generate stress testing configuration
mcptesta generate-config stress stress_test.yaml \
  --parallel-workers 16 \
  --test-types "tool_call,resource_read"

Running Generated Templates

# Run basic configuration
mcptesta yaml my_basic_config.yaml

# Run with custom settings
mcptesta yaml advanced_config.yaml --parallel 12 --output ./results

# Run specific test categories
mcptesta yaml stress_test.yaml --tag performance --tag load

Template Features Comparison

Feature	Basic	Intermediate	Advanced	Expert	Stress	Integration
Complexity	Low	Medium	High	Very High	High	High
Parallel Workers	2	4	6	12	16	8
Test Suites	3	7	12	15+	10	8
MCP Protocol Features	Basic	Some	All	All+	Some	All
Multi-Server	❌	✅	✅	✅	✅	✅
Dependencies	❌	✅	✅	✅	❌	✅
Performance Testing	❌	❌	✅	✅	✅	✅
External Systems	❌	❌	✅	✅	❌	✅
Chaos Engineering	❌	❌	❌	✅	✅	❌
Documentation Level	High	High	Medium	Low	Medium	Medium

Customization Guide

1. Server Configuration

Update server commands to match your FastMCP server:

servers:
  - name: "my_server"
    command: "python -m my_fastmcp_server"  # Update this
    transport: "stdio"
    timeout: 30

2. Test Parameters

Modify test parameters to match your server's API:

- name: "my_tool_test"
  test_type: "tool_call"
  target: "echo"  # Change to your tool name
  parameters:
    message: "test"  # Update parameters

3. Expected Results

Define what you expect from your server:

expected:
  message: "test"  # What your server should return
  status: "success"

4. Environment Variables

Use variables for flexible configuration:

variables:
  SERVER_COMMAND: "python -m my_server"
  TEST_MESSAGE: "Hello, World!"
  TIMEOUT: "30"

Best Practices

1. Start Simple

• Begin with the basic template
• Gradually move to more complex templates as you learn

2. Incremental Complexity

• Don't jump straight to expert level
• Add features progressively based on your needs

3. Test Organization

• Use meaningful test names and descriptions
• Group related tests in suites
• Use tags for easy filtering

4. Error Handling

• Include negative test cases
• Test error conditions explicitly
• Validate error messages and types

5. Performance Considerations

• Monitor resource usage during testing
• Use appropriate parallel worker counts
• Include performance baselines

Template Development

Creating Custom Templates

For specific use cases, you can create custom templates by:

1. Starting with the closest existing template
2. Modifying test cases for your scenario
3. Adjusting configuration parameters
4. Adding custom variables and documentation

Contributing Templates

If you develop useful templates for specific scenarios:

1. Document the use case clearly
2. Include comprehensive examples
3. Test thoroughly with different servers
4. Consider contributing back to the project

Troubleshooting

Common Issues

1. Server Connection Failures

   • Check server command and transport type
   • Verify server starts correctly
   • Check timeout settings

2. Test Failures

   • Verify tool/resource names match your server
   • Check parameter formats and types
   • Review expected results

3. Performance Issues

   • Reduce parallel worker count
   • Increase timeout values
   • Check system resources

4. Template Errors

   • Validate YAML syntax
   • Check variable references
   • Verify dependencies exist

Getting Help

• Use mcptesta yaml config.yaml --dry-run to validate configurations
• Use mcptesta yaml config.yaml --list-tests to see what will run
• Check the MCPTesta documentation for detailed guidance
• Review server logs for connection and execution issues

Template Evolution

These templates evolve based on:

• MCPTesta feature additions
• Community feedback and contributions
• Best practices discovery
• Real-world usage patterns

Stay updated with the latest templates and contribute your own improvements!