pypi-query-mcp/EXTRAS_FIX_SUMMARY.md

Fix for include_extras Parameter Validation Issue

Summary

Fixed a critical issue where the include_extras parameter in the resolve_dependencies tool was not working correctly due to incorrect Python version filtering that was removing extra dependencies from consideration.

Root Cause Analysis

The issue was in the _is_requirement_applicable method in pypi_query_mcp/core/dependency_parser.py. When filtering requirements by Python version, the method was evaluating markers like extra == "socks" in an environment that didn't include the extra variable, causing these requirements to be filtered out incorrectly.

The Problem

1. Python version filtering was too aggressive: The filter_requirements_by_python_version method was filtering out ALL requirements with markers that couldn't be evaluated, including extra requirements.

2. Marker evaluation environment was incomplete: When evaluating markers like extra == "socks", the environment didn't include the extra variable, so the evaluation always returned False.

3. Incorrect filtering logic: Extra dependencies should not be filtered by Python version at all - they should be handled separately based on user selection.

Before the Fix

# In _is_requirement_applicable method
def _is_requirement_applicable(self, req: Requirement, python_version: Version) -> bool:
    if not req.marker:
        return True

    env = {
        "python_version": str(python_version),
        # ... other environment variables
    }
    
    try:
        return req.marker.evaluate(env)  # This returns False for extra == "socks"
    except Exception as e:
        logger.warning(f"Failed to evaluate marker for {req}: {e}")
        return True

Result: Requirements like PySocks!=1.5.7,>=1.5.6; extra == "socks" were filtered out because extra == "socks" evaluated to False in an environment without the extra variable.

The Fix

Added a check to exclude extra requirements from Python version filtering:

def _is_requirement_applicable(self, req: Requirement, python_version: Version) -> bool:
    if not req.marker:
        return True

    # If the marker contains 'extra ==', this is an extra dependency
    # and should not be filtered by Python version. Extra dependencies
    # are handled separately based on user selection.
    marker_str = str(req.marker)
    if "extra ==" in marker_str:
        return True

    # Rest of the method unchanged...

Why This Fix Works

1. Preserves extra requirements: Extra dependencies are no longer filtered out by Python version filtering
2. Maintains correct Python version filtering: Non-extra requirements are still properly filtered by Python version
3. Separates concerns: Extra handling is kept separate from Python version filtering, as it should be
4. Backwards compatible: Doesn't break existing functionality

Testing Results

Before the Fix

# Example: requests[socks] with Python 3.10
result = await resolve_package_dependencies("requests", include_extras=["socks"], python_version="3.10")
print(result["summary"]["total_extra_dependencies"])  # Output: 0 ❌

After the Fix

# Example: requests[socks] with Python 3.10  
result = await resolve_package_dependencies("requests", include_extras=["socks"], python_version="3.10")
print(result["summary"]["total_extra_dependencies"])  # Output: 1 ✅
print(result["dependency_tree"]["requests"]["dependencies"]["extras"])
# Output: {'socks': ['PySocks!=1.5.7,>=1.5.6; extra == "socks"']} ✅

Examples of Correct Usage

Basic Examples

# Requests with SOCKS proxy support
await resolve_package_dependencies("requests", include_extras=["socks"])

# Django with password hashing extras
await resolve_package_dependencies("django", include_extras=["argon2", "bcrypt"])

# Setuptools with testing tools
await resolve_package_dependencies("setuptools", include_extras=["test"])

# Flask with async and dotenv support
await resolve_package_dependencies("flask", include_extras=["async", "dotenv"])

How to Find Available Extras

1. Check the package's PyPI page - Look for available extras in the description
2. Use the provides_extra field - Available in package metadata
3. Check package documentation - Often lists available extras
4. Look for requirements with extra == - In the requires_dist field

Common Mistakes to Avoid

❌ Wrong: Using generic extra names

# These don't exist for requests
await resolve_package_dependencies("requests", include_extras=["dev", "test"])

✅ Right: Using package-specific extra names

# These are actual extras for requests
await resolve_package_dependencies("requests", include_extras=["socks", "use-chardet-on-py3"])

Files Modified

1. pypi_query_mcp/core/dependency_parser.py: Fixed _is_requirement_applicable method
2. pypi_query_mcp/server.py: Updated documentation for include_extras parameter
3. pypi_query_mcp/tools/dependency_resolver.py: Updated docstring for include_extras parameter
4. tests/test_dependency_resolver.py: Added comprehensive test cases
5. examples/extras_usage_demo.py: Added demonstration of correct usage

Validation

The fix has been validated with:

• ✅ Real PyPI packages (requests, django, setuptools, flask)
• ✅ Various extra combinations
• ✅ Python version filtering still works for non-extra requirements
• ✅ Transitive dependency resolution with extras
• ✅ Edge cases (non-existent extras, extras with no dependencies)
• ✅ Backwards compatibility with existing code

Performance Impact

• Minimal: Only adds a simple string check for "extra ==" in markers
• Positive: Reduces unnecessary marker evaluations for extra requirements
• No regression: All existing functionality continues to work as before

Conclusion

This fix resolves the core issue with include_extras parameter validation while maintaining all existing functionality. Users can now successfully resolve optional dependencies using the correct extra names as defined by each package.