pypi-query-mcp/EXTRAS_FIX_SUMMARY.md

# Fix for include_extras Parameter Validation Issue

## Summary

Fixed a critical issue where the `include_extras` parameter in the `resolve_dependencies` tool was not working correctly due to incorrect Python version filtering that was removing extra dependencies from consideration.

## Root Cause Analysis

The issue was in the `_is_requirement_applicable` method in `pypi_query_mcp/core/dependency_parser.py`. When filtering requirements by Python version, the method was evaluating markers like `extra == "socks"` in an environment that didn't include the `extra` variable, causing these requirements to be filtered out incorrectly.

### The Problem

1. **Python version filtering was too aggressive**: The `filter_requirements_by_python_version` method was filtering out ALL requirements with markers that couldn't be evaluated, including extra requirements.

2. **Marker evaluation environment was incomplete**: When evaluating markers like `extra == "socks"`, the environment didn't include the `extra` variable, so the evaluation always returned `False`.

3. **Incorrect filtering logic**: Extra dependencies should not be filtered by Python version at all - they should be handled separately based on user selection.

### Before the Fix

```python
# In _is_requirement_applicable method
def _is_requirement_applicable(self, req: Requirement, python_version: Version) -> bool:
    if not req.marker:
        return True

    env = {
        "python_version": str(python_version),
        # ... other environment variables
    }
    
    try:
        return req.marker.evaluate(env)  # This returns False for extra == "socks"
    except Exception as e:
        logger.warning(f"Failed to evaluate marker for {req}: {e}")
        return True
```

**Result**: Requirements like `PySocks!=1.5.7,>=1.5.6; extra == "socks"` were filtered out because `extra == "socks"` evaluated to `False` in an environment without the `extra` variable.

## The Fix

Added a check to exclude extra requirements from Python version filtering:

```python
def _is_requirement_applicable(self, req: Requirement, python_version: Version) -> bool:
    if not req.marker:
        return True

    # If the marker contains 'extra ==', this is an extra dependency
    # and should not be filtered by Python version. Extra dependencies
    # are handled separately based on user selection.
    marker_str = str(req.marker)
    if "extra ==" in marker_str:
        return True

    # Rest of the method unchanged...
```

### Why This Fix Works

1. **Preserves extra requirements**: Extra dependencies are no longer filtered out by Python version filtering
2. **Maintains correct Python version filtering**: Non-extra requirements are still properly filtered by Python version
3. **Separates concerns**: Extra handling is kept separate from Python version filtering, as it should be
4. **Backwards compatible**: Doesn't break existing functionality

## Testing Results

### Before the Fix
```python
# Example: requests[socks] with Python 3.10
result = await resolve_package_dependencies("requests", include_extras=["socks"], python_version="3.10")
print(result["summary"]["total_extra_dependencies"])  # Output: 0 ❌
```

### After the Fix
```python
# Example: requests[socks] with Python 3.10  
result = await resolve_package_dependencies("requests", include_extras=["socks"], python_version="3.10")
print(result["summary"]["total_extra_dependencies"])  # Output: 1 ✅
print(result["dependency_tree"]["requests"]["dependencies"]["extras"])
# Output: {'socks': ['PySocks!=1.5.7,>=1.5.6; extra == "socks"']} ✅
```

## Examples of Correct Usage

### Basic Examples

```python
# Requests with SOCKS proxy support
await resolve_package_dependencies("requests", include_extras=["socks"])

# Django with password hashing extras
await resolve_package_dependencies("django", include_extras=["argon2", "bcrypt"])

# Setuptools with testing tools
await resolve_package_dependencies("setuptools", include_extras=["test"])

# Flask with async and dotenv support
await resolve_package_dependencies("flask", include_extras=["async", "dotenv"])
```

### How to Find Available Extras

1. **Check the package's PyPI page** - Look for available extras in the description
2. **Use the `provides_extra` field** - Available in package metadata
3. **Check package documentation** - Often lists available extras
4. **Look for requirements with `extra ==`** - In the `requires_dist` field

### Common Mistakes to Avoid

❌ **Wrong**: Using generic extra names
```python
# These don't exist for requests
await resolve_package_dependencies("requests", include_extras=["dev", "test"])
```

✅ **Right**: Using package-specific extra names
```python
# These are actual extras for requests
await resolve_package_dependencies("requests", include_extras=["socks", "use-chardet-on-py3"])
```

## Files Modified

1. **`pypi_query_mcp/core/dependency_parser.py`**: Fixed `_is_requirement_applicable` method
2. **`pypi_query_mcp/server.py`**: Updated documentation for `include_extras` parameter
3. **`pypi_query_mcp/tools/dependency_resolver.py`**: Updated docstring for `include_extras` parameter
4. **`tests/test_dependency_resolver.py`**: Added comprehensive test cases
5. **`examples/extras_usage_demo.py`**: Added demonstration of correct usage

## Validation

The fix has been validated with:

- ✅ Real PyPI packages (requests, django, setuptools, flask)
- ✅ Various extra combinations
- ✅ Python version filtering still works for non-extra requirements  
- ✅ Transitive dependency resolution with extras
- ✅ Edge cases (non-existent extras, extras with no dependencies)
- ✅ Backwards compatibility with existing code

## Performance Impact

- **Minimal**: Only adds a simple string check for `"extra =="` in markers
- **Positive**: Reduces unnecessary marker evaluations for extra requirements
- **No regression**: All existing functionality continues to work as before

## Conclusion

This fix resolves the core issue with `include_extras` parameter validation while maintaining all existing functionality. Users can now successfully resolve optional dependencies using the correct extra names as defined by each package.