mcptesta/examples/templates/README.md

# MCPTesta Configuration Templates

This directory contains comprehensive configuration templates for MCPTesta, demonstrating different complexity levels and use cases.

## Available Templates

### 📚 Basic Template (`basic_template.yaml`)
**Perfect for beginners and quick validation**
- Simple server configuration
- Basic connectivity and tool testing
- Console output with clear documentation
- Essential features only

**Use when:**
- Learning MCPTesta fundamentals
- Quick server validation
- Simple testing scenarios

### 📈 Intermediate Template (`intermediate_template.yaml`)  
**Mid-level template with advanced features**
- Multiple test suites with dependencies
- Basic MCP protocol features (notifications, progress)
- HTML reporting and environment variables
- Error handling and retry policies

**Use when:**
- Need dependency management between tests
- Want to explore MCP protocol features
- Require organized test suites with setup/teardown

### 🚀 Advanced Template (`advanced_template.yaml`)
**Full-featured template with all MCP capabilities**
- Complete MCP protocol feature testing
- Multi-server coordination
- Performance testing and profiling
- Complex scenarios and error handling

**Use when:**
- Production-ready testing
- Need comprehensive MCP protocol coverage
- Testing complex multi-server scenarios

### 🎯 Expert Template (`expert_template.yaml`)
**Maximum complexity for expert users**
- Distributed system testing
- Chaos engineering patterns
- Advanced authentication schemes
- Enterprise-grade features

**Use when:**
- Expert-level protocol testing
- Distributed system validation
- Chaos engineering and resilience testing
- Protocol development and research

### ⚡ Stress Template (`stress_template.yaml`)
**Specialized performance and load testing**
- Various load patterns and stress scenarios
- Resource exhaustion testing
- Performance benchmarking
- Long-duration stability testing

**Use when:**
- Performance validation
- Load testing and capacity planning
- Identifying bottlenecks and limits
- Stability and endurance testing

### 🔗 Integration Template (`integration_template.yaml`)
**Multi-service and external system integration**
- End-to-end workflow testing
- Service mesh and discovery
- External system integration
- Transaction management

**Use when:**
- Testing multi-service architectures
- Validating service interactions
- External system integration
- Workflow orchestration testing

## Usage Examples

### Basic Usage
```bash
# Generate a basic template
mcptesta generate-config basic my_basic_config.yaml

# Generate with custom server command
mcptesta generate-config basic my_config.yaml --server-command "uvx my-server"
```

### Advanced Usage
```bash
# Generate advanced template with custom features
mcptesta generate-config advanced advanced_config.yaml \
  --server-command "python -m my_server --production" \
  --parallel-workers 8 \
  --enable-features "notifications,progress,cancellation"

# Generate stress testing configuration
mcptesta generate-config stress stress_test.yaml \
  --parallel-workers 16 \
  --test-types "tool_call,resource_read"
```

### Running Generated Templates
```bash
# Run basic configuration
mcptesta yaml my_basic_config.yaml

# Run with custom settings
mcptesta yaml advanced_config.yaml --parallel 12 --output ./results

# Run specific test categories
mcptesta yaml stress_test.yaml --tag performance --tag load
```

## Template Features Comparison

| Feature | Basic | Intermediate | Advanced | Expert | Stress | Integration |
|---------|-------|--------------|----------|---------|---------|-------------|
| **Complexity** | Low | Medium | High | Very High | High | High |
| **Parallel Workers** | 2 | 4 | 6 | 12 | 16 | 8 |
| **Test Suites** | 3 | 7 | 12 | 15+ | 10 | 8 |
| **MCP Protocol Features** | Basic | Some | All | All+ | Some | All |
| **Multi-Server** | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ |
| **Dependencies** | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ |
| **Performance Testing** | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ |
| **External Systems** | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ |
| **Chaos Engineering** | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ |
| **Documentation Level** | High | High | Medium | Low | Medium | Medium |

## Customization Guide

### 1. Server Configuration
Update server commands to match your FastMCP server:
```yaml
servers:
  - name: "my_server"
    command: "python -m my_fastmcp_server"  # Update this
    transport: "stdio"
    timeout: 30
```

### 2. Test Parameters
Modify test parameters to match your server's API:
```yaml
- name: "my_tool_test"
  test_type: "tool_call"
  target: "echo"  # Change to your tool name
  parameters:
    message: "test"  # Update parameters
```

### 3. Expected Results
Define what you expect from your server:
```yaml
expected:
  message: "test"  # What your server should return
  status: "success"
```

### 4. Environment Variables
Use variables for flexible configuration:
```yaml
variables:
  SERVER_COMMAND: "python -m my_server"
  TEST_MESSAGE: "Hello, World!"
  TIMEOUT: "30"
```

## Best Practices

### 1. Start Simple
- Begin with the basic template
- Gradually move to more complex templates as you learn

### 2. Incremental Complexity
- Don't jump straight to expert level
- Add features progressively based on your needs

### 3. Test Organization
- Use meaningful test names and descriptions
- Group related tests in suites
- Use tags for easy filtering

### 4. Error Handling
- Include negative test cases
- Test error conditions explicitly
- Validate error messages and types

### 5. Performance Considerations
- Monitor resource usage during testing
- Use appropriate parallel worker counts
- Include performance baselines

## Template Development

### Creating Custom Templates
For specific use cases, you can create custom templates by:

1. Starting with the closest existing template
2. Modifying test cases for your scenario
3. Adjusting configuration parameters
4. Adding custom variables and documentation

### Contributing Templates
If you develop useful templates for specific scenarios:

1. Document the use case clearly
2. Include comprehensive examples
3. Test thoroughly with different servers
4. Consider contributing back to the project

## Troubleshooting

### Common Issues

1. **Server Connection Failures**
   - Check server command and transport type
   - Verify server starts correctly
   - Check timeout settings

2. **Test Failures**
   - Verify tool/resource names match your server
   - Check parameter formats and types
   - Review expected results

3. **Performance Issues**
   - Reduce parallel worker count
   - Increase timeout values
   - Check system resources

4. **Template Errors**
   - Validate YAML syntax
   - Check variable references
   - Verify dependencies exist

### Getting Help

- Use `mcptesta yaml config.yaml --dry-run` to validate configurations
- Use `mcptesta yaml config.yaml --list-tests` to see what will run
- Check the MCPTesta documentation for detailed guidance
- Review server logs for connection and execution issues

## Template Evolution

These templates evolve based on:
- MCPTesta feature additions
- Community feedback and contributions  
- Best practices discovery
- Real-world usage patterns

Stay updated with the latest templates and contribute your own improvements!