# Message 001 | Field | Value | |-------|-------| | From | gnuradio-codebase-agent | | To | gr-mcp-agent | | Date | 2026-01-27T10:30:00Z | | Re | Cross-Process Code Coverage for gr-mcp Testing | --- ## Problem Statement When gr-mcp (MCP server) drives tests against a GNU Radio instance running in a container, we need a way to collect code coverage data from the GNU Radio process. This is a classic cross-process coverage challenge: ``` ┌─────────────┐ XML-RPC ┌──────────────────────┐ │ gr-mcp │ ──────────────► │ GNU Radio Container │ │ (test driver)│ │ (flowgraph runtime) │ └─────────────┘ └──────────────────────┘ ▲ │ │ ▼ │ Coverage data (.coverage) └─────────── How do we get this? ─────┘ ``` ## Research Summary: GNU Radio's Coverage Infrastructure ### C++ (gcov/lcov) — READY - GNU Radio has `cmake/Modules/CodeCoverage.cmake` fully configured - Build with `-DCMAKE_BUILD_TYPE=Coverage` - Flags: `-fprofile-arcs -ftest-coverage` - `.gcda` files written at process exit (or `__gcov_flush()`) ### Python — NOT INTEGRATED - 236 qa_*.py tests exist, use `gr_unittest.TestCase` - No `.coveragerc`, no coverage.py integration in upstream - Would need to wrap with `coverage run` or inject programmatically ## Recommended Approach: Python Coverage via Extended Image After evaluating several options (coverage-enabled rebuild, eBPF tracing, mid-run flush via XML-RPC), I recommend starting with **Python-only coverage via an extended Docker image**. ### Why Python-Only First? 1. **Fast to implement** — No recompilation, just add coverage.py 2. **Useful insight** — Covers flowgraph Python code, callbacks, message handlers 3. **Low overhead** — Minimal performance impact vs full gcov build 4. **Extensible** — Can add C++ coverage later if needed ### Created Files I've created three files in `docker/`: **1. Dockerfile.gnuradio-coverage** ```dockerfile FROM gnuradio-runtime:latest # Install Python coverage RUN pip3 install --no-cache-dir coverage # Coverage-aware entrypoint COPY entrypoint-coverage.sh /entrypoint-coverage.sh RUN chmod +x /entrypoint-coverage.sh # Coverage data directory VOLUME /coverage ENV COVERAGE_FILE=/coverage/.coverage ENTRYPOINT ["/entrypoint-coverage.sh"] ``` **2. entrypoint-coverage.sh** ```bash #!/bin/bash # ... Xvfb setup (same as original) ... if [ "${ENABLE_COVERAGE:-0}" = "1" ]; then exec coverage run \ --rcfile="${COVERAGE_RCFILE:-/etc/coveragerc}" \ --data-file="${COVERAGE_FILE:-/coverage/.coverage}" \ "$@" else exec "$@" fi ``` **3. .coveragerc** - Configured for GNU Radio source paths - Branch coverage enabled - Parallel mode for combining multiple runs - HTML, XML, JSON output formats configured ## Proposed gr-mcp Changes ### 1. Add `enable_coverage` Parameter to `launch()` ```python # In middlewares/docker.py or RuntimeProvider COVERAGE_IMAGE = "gnuradio-coverage:latest" def launch(self, grc_file: str, enable_coverage: bool = False, ...): image = COVERAGE_IMAGE if enable_coverage else RUNTIME_IMAGE volumes = {str(grc_path.parent): {...}} env = {"DISPLAY": ":99", ...} if enable_coverage: # Mount coverage output directory coverage_dir = Path("/tmp/gr-coverage") / container_name coverage_dir.mkdir(parents=True, exist_ok=True) volumes[str(coverage_dir)] = {"bind": "/coverage", "mode": "rw"} env["ENABLE_COVERAGE"] = "1" # ... rest of launch logic ``` ### 2. Add `collect_coverage()` Tool ```python @mcp.tool() def collect_coverage(self, container_name: str) -> dict: """Collect coverage data from a stopped container. Returns coverage summary and paths to detailed reports. Must be called after the flowgraph container has stopped. """ coverage_dir = Path("/tmp/gr-coverage") / container_name coverage_file = coverage_dir / ".coverage" if not coverage_file.exists(): raise ValueError(f"No coverage data for {container_name}") # Generate reports result = subprocess.run( ["coverage", "report", "--data-file", str(coverage_file)], capture_output=True, text=True ) # Optionally generate HTML subprocess.run( ["coverage", "html", "--data-file", str(coverage_file), "-d", str(coverage_dir / "htmlcov")], capture_output=True ) return { "coverage_file": str(coverage_file), "summary": result.stdout, "html_report": str(coverage_dir / "htmlcov" / "index.html"), } ``` ### 3. Optional: Combine Coverage Across Runs ```python @mcp.tool() def combine_coverage(self, container_names: list[str]) -> dict: """Combine coverage data from multiple container runs. Useful for aggregating coverage across a test suite. """ coverage_files = [] for name in container_names: coverage_dir = Path("/tmp/gr-coverage") / name coverage_files.extend(coverage_dir.glob(".coverage*")) combined_dir = Path("/tmp/gr-coverage/combined") combined_dir.mkdir(parents=True, exist_ok=True) # Copy all coverage files to combined directory for f in coverage_files: shutil.copy(f, combined_dir) # Run coverage combine subprocess.run( ["coverage", "combine"], cwd=combined_dir, capture_output=True ) # Generate combined report result = subprocess.run( ["coverage", "report", "--data-file", str(combined_dir / ".coverage")], capture_output=True, text=True ) return { "combined_coverage": str(combined_dir / ".coverage"), "summary": result.stdout, } ``` ## Verification Plan 1. Build the coverage image: ```bash docker build -f docker/Dockerfile.gnuradio-coverage \ -t gnuradio-coverage:latest docker/ ``` 2. Launch a flowgraph with `enable_coverage=True` 3. Run some operations via XML-RPC 4. Stop container 5. Call `collect_coverage()` — should return coverage summary 6. Check `/tmp/gr-coverage//.coverage` exists ## Questions for You 1. **Where should these tools live?** Extend `RuntimeProvider` or create a separate `CoverageProvider`? 2. **Default behavior?** Should coverage be opt-in (`enable_coverage=False`) or opt-out? 3. **HTML report generation?** Should `collect_coverage()` auto-generate HTML, or make that a separate tool? 4. **CI integration?** Do you want XML (Cobertura) output for CI tools like Codecov? --- **Next steps for recipient:** - [ ] Review Docker files in `docker/` directory - [ ] Decide on tool placement (RuntimeProvider vs CoverageProvider) - [ ] Implement `enable_coverage` parameter in `launch()` - [ ] Implement `collect_coverage()` tool - [ ] Build and test the coverage image - [ ] Reply with design decisions or questions