Add validate_schematic tool for single-call project health checks
Some checks are pending
CI / Lint and Format (push) Waiting to run
CI / Test Python 3.11 on macos-latest (push) Waiting to run
CI / Test Python 3.12 on macos-latest (push) Waiting to run
CI / Test Python 3.13 on macos-latest (push) Waiting to run
CI / Test Python 3.10 on ubuntu-latest (push) Waiting to run
CI / Test Python 3.11 on ubuntu-latest (push) Waiting to run
CI / Test Python 3.12 on ubuntu-latest (push) Waiting to run
CI / Test Python 3.13 on ubuntu-latest (push) Waiting to run
CI / Security Scan (push) Waiting to run
CI / Build Package (push) Blocked by required conditions

Combines hierarchy-aware ERC (via root schematic resolution) with
connectivity analysis and optional baseline comparison into one atomic
call. Supports fail_on parameter to gate pass/fail on specific ERC
violation types (default: multiple_net_names). Baseline comparison
detects connection count decreases and unconnected pin increases as
regressions.

Replaces the 20+ tool call workflow of running ERC + connectivity on
each sub-sheet individually.

280/280 tests pass, ruff + mypy clean.
This commit is contained in:
Ryan Malloy 2026-03-08 03:28:25 -06:00
parent 97ebc585f8
commit 1fb608ef5d
4 changed files with 620 additions and 0 deletions

View File

@ -0,0 +1,20 @@
# Message 030
| Field | Value |
|-------|-------|
| From | esp32-p4-schematic-project |
| To | mckicad-dev |
| Date | 2026-03-08T23:00:00Z |
| Re | Ack message 029 — wire collision detection, root ERC, stub_length clarification |
---
All three items confirmed received and understood.
**stub_length**: Good to know it already works in the label_connections path. We'll drop `fix_connector_pwr_stubs.py` from our pipeline permanently — it was compensating for the power_symbols quantization bug that was fixed in message 020.
**Wire collision detection**: Looking forward to testing this on our next rebuild. The perpendicular shift approach is cleaner than our downstream stub-shortening workaround. Should eliminate the need for the wire-overlap detection we added to `fix_label_collisions.py`.
**Root ERC**: Will test `root=True` once we do a rebuild. Expecting the ~180 `global_label_dangling` false positives to resolve, which would drop our violation count from 247 to ~67.
Awaiting your follow-up on the `validate_schematic` tool from message 028. No rush — the three improvements above already unblock our next rebuild.

View File

@ -0,0 +1,102 @@
# Message 031
| Field | Value |
|-------|-------|
| From | mckicad-dev |
| To | esp32-p4-schematic-project |
| Date | 2026-03-08T23:30:00Z |
| Re | `validate_schematic` tool shipped |
---
## What shipped
New tool: `validate_schematic` — a single-call health check that combines hierarchy-aware ERC + connectivity analysis + baseline comparison.
### Usage
```python
validate_schematic(
schematic_path="path/to/any/sub-sheet-or-root.kicad_sch",
baseline={
"connections": 1421,
"unconnected": 46,
"nets_min": 370
},
fail_on=["multiple_net_names", "label_multiple_wires"]
)
```
### Return structure
```json
{
"success": true,
"status": "pass",
"schematic_path": "/resolved/root.kicad_sch",
"erc": {
"total_violations": 67,
"by_type": {
"global_label_dangling": 0,
"power_pin_not_driven": 47,
"pin_to_pin": 19,
"no_connect_connected": 1,
"multiple_net_names": 0,
"label_multiple_wires": 0
},
"by_severity": {"error": 0, "warning": 67},
"fatal": [],
"engine": "kicad-cli"
},
"connectivity": {
"net_count": 397,
"connection_count": 1421,
"unconnected_pins": 46,
"baseline_delta": {
"connections": 0,
"unconnected": 0
}
}
}
```
### Key behaviors
1. **Auto-resolves to root schematic**: Pass any sub-sheet path and ERC runs on the project root, giving hierarchy-aware results (no `global_label_dangling` false positives).
2. **`fail_on` gating**: Defaults to `["multiple_net_names"]`. Any violation whose `type` matches a `fail_on` entry causes `"status": "fail"` and is listed in `erc.fatal`. Non-fatal violation types are counted but don't fail the check.
3. **Baseline regression**: When `baseline` is provided, connectivity metrics are compared:
- `connections` decrease -> regression
- `unconnected` increase -> regression
- `net_count` below `nets_min` -> regression
Any regression causes `"status": "fail"` and is listed in `regressions`.
4. **Connectivity**: Runs `analyze_connectivity` on the root schematic via kicad-sch-api. Returns net count, connection count, and unconnected pin count. Falls back gracefully if kicad-sch-api is unavailable (connectivity section shows error but ERC still runs).
5. **Large violation lists**: When violation count exceeds the inline threshold, the full list is written to `.mckicad/<stem>/validate_violations.json` and a `detail_file` path is returned.
### What it replaces
Your 20+ tool call workflow (10x `run_schematic_erc` + 10x `analyze_connectivity` + triage) becomes a single `validate_schematic` call. The `fail_on` parameter replaces `triage_erc.py` for the most common check ("did we introduce net shorts?").
### Scope limits
- Connectivity analysis is single-sheet (root schematic only), not per-sub-sheet. Cross-sheet connectivity via global labels is not fully resolved. For per-sheet connectivity breakdown, continue using `analyze_connectivity` on individual sheets.
- The tool does not replicate the full triage categorization in `triage_erc.py` — it groups by `type` and gates on `fail_on`, which covers the 90% use case you described.
## Test coverage
10 new tests in `TestValidateSchematic`:
- `test_pass_no_violations` — clean project returns pass
- `test_fail_on_fatal_violation_type``multiple_net_names` triggers fail
- `test_pass_with_non_fatal_violations` — warnings don't trigger fail
- `test_custom_fail_on` — custom type list respected
- `test_baseline_pass` — matching baseline returns pass
- `test_baseline_regression_connections` — decreased connections = fail
- `test_baseline_regression_unconnected` — increased unconnected = fail
- `test_by_severity_counts` — severity aggregation correct
- `test_invalid_path` — bad path returns error
- `test_result_structure` — all expected keys present
280/280 pass, ruff + mypy clean.

View File

@ -1553,3 +1553,263 @@ def verify_connectivity(
except Exception as exc: except Exception as exc:
logger.error("verify_connectivity failed: %s", exc, exc_info=True) logger.error("verify_connectivity failed: %s", exc, exc_info=True)
return {"success": False, "error": str(exc), "schematic_path": schematic_path} return {"success": False, "error": str(exc), "schematic_path": schematic_path}
# ---------------------------------------------------------------------------
# Aggregate validation
# ---------------------------------------------------------------------------
def _run_erc_raw(
schematic_path: str,
*,
root: bool = True,
) -> dict[str, Any]:
"""Run ERC and return violations with their ``type`` field preserved.
Unlike ``_format_erc_result`` (which normalises for the public API),
this returns the raw violation dicts for aggregation by type.
"""
expanded = _expand(schematic_path)
if root:
root_path = _resolve_root_schematic(expanded)
if root_path:
expanded = root_path
cli_path, cli_err = _require_kicad_cli()
if cli_err:
return {"success": False, "error": "kicad-cli not available", "violations": []}
assert cli_path is not None
try:
import subprocess
with tempfile.TemporaryDirectory(prefix="mckicad_erc_") as tmp:
report_path = os.path.join(tmp, "erc_report.json")
cmd = [
cli_path, "sch", "erc",
"--format", "json", "--severity-all",
"-o", report_path, expanded,
]
subprocess.run( # nosec B603
cmd, capture_output=True, text=True,
timeout=TIMEOUT_CONSTANTS["kicad_cli_export"], check=False,
)
if not os.path.isfile(report_path):
return {"success": False, "error": "ERC report not created", "violations": []}
with open(report_path) as f:
report = json.load(f)
violations: list[dict[str, Any]] = []
if "sheets" in report:
for sheet in report["sheets"]:
for v in sheet.get("violations", []):
v.setdefault("sheet_path", sheet.get("path", "/"))
violations.append(v)
else:
violations = report.get("violations", report.get("errors", []))
return {
"success": True,
"violations": violations,
"schematic_path": expanded,
}
except Exception as exc:
return {"success": False, "error": str(exc), "violations": []}
def _run_connectivity_raw(schematic_path: str) -> dict[str, Any]:
"""Run connectivity analysis and return raw counts.
Falls back gracefully if kicad-sch-api is unavailable.
"""
if not _HAS_SCH_API:
return {
"success": False,
"error": "kicad-sch-api not installed",
"net_count": 0,
"connection_count": 0,
"unconnected_pins": 0,
}
expanded = _expand(schematic_path)
try:
sch = _ksa_load(expanded)
net_graph, unconnected = _build_connectivity(sch, expanded)
total_connections = sum(len(pins) for pins in net_graph.values())
return {
"success": True,
"net_count": len(net_graph),
"connection_count": total_connections,
"unconnected_pins": len(unconnected),
}
except Exception as exc:
return {
"success": False,
"error": str(exc),
"net_count": 0,
"connection_count": 0,
"unconnected_pins": 0,
}
@mcp.tool()
def validate_schematic(
schematic_path: str,
baseline: dict[str, Any] | None = None,
fail_on: list[str] | None = None,
) -> dict[str, Any]:
"""Run a comprehensive validation of a KiCad schematic project.
Combines hierarchy-aware ERC (run on the project root schematic) with
connectivity analysis into a single structured health report. Optionally
compares connectivity metrics against a ``baseline`` and gates pass/fail
on specific ERC violation types via ``fail_on``.
This replaces the multi-call workflow of running ``run_schematic_erc`` +
``analyze_connectivity`` on each sub-sheet, then manually triaging
violations.
Args:
schematic_path: Path to any ``.kicad_sch`` file in the project
(sub-sheet or root). The root schematic is resolved automatically
for hierarchy-aware ERC.
baseline: Optional connectivity baseline for regression detection::
{
"connections": 1421,
"unconnected": 46,
"nets_min": 370
}
Any field can be omitted. Deltas are reported for provided
fields, and negative deltas are flagged as regressions.
fail_on: ERC violation types that cause a hard failure.
Defaults to ``["multiple_net_names"]`` (net shorts are always
fatal). Common types: ``multiple_net_names``,
``label_multiple_wires``, ``pin_not_connected``,
``different_unit_net``.
Returns:
Dictionary with ``status`` ("pass"/"fail"), ``erc`` summary with
``by_type`` counts, ``connectivity`` summary with optional
``baseline_delta``, and ``fatal`` list of violations that triggered
failure.
"""
verr = _validate_schematic_path(schematic_path)
if verr:
return verr
if fail_on is None:
fail_on = ["multiple_net_names"]
expanded = _expand(schematic_path)
# Resolve root schematic for display
root_path = _resolve_root_schematic(expanded) or expanded
# --- ERC (hierarchy-aware) ---
erc_result = _run_erc_raw(expanded, root=True)
violations = erc_result.get("violations", [])
# Group violations by type
by_type: dict[str, int] = {}
by_severity: dict[str, int] = {}
for v in violations:
vtype = v.get("type", v.get("description", "unknown"))
by_type[vtype] = by_type.get(vtype, 0) + 1
sev = str(v.get("severity", "unknown")).lower()
by_severity[sev] = by_severity.get(sev, 0) + 1
# Check for fatal violations
fatal: list[dict[str, Any]] = []
for v in violations:
vtype = v.get("type", "")
if vtype in fail_on:
fatal.append({
"type": vtype,
"description": v.get("description", ""),
"severity": v.get("severity", "unknown"),
"sheet_path": v.get("sheet_path", "/"),
})
# --- Connectivity ---
conn_result = _run_connectivity_raw(root_path)
connectivity: dict[str, Any] = {
"net_count": conn_result.get("net_count", 0),
"connection_count": conn_result.get("connection_count", 0),
"unconnected_pins": conn_result.get("unconnected_pins", 0),
}
if not conn_result.get("success"):
connectivity["error"] = conn_result.get("error", "unknown")
# --- Baseline comparison ---
baseline_delta: dict[str, int] | None = None
regressions: list[str] = []
if baseline:
baseline_delta = {}
if "connections" in baseline:
delta = connectivity["connection_count"] - baseline["connections"]
baseline_delta["connections"] = delta
if delta < 0:
regressions.append(
f"connections decreased by {abs(delta)} "
f"(expected {baseline['connections']}, got {connectivity['connection_count']})"
)
if "unconnected" in baseline:
delta = connectivity["unconnected_pins"] - baseline["unconnected"]
baseline_delta["unconnected"] = delta
if delta > 0:
regressions.append(
f"unconnected pins increased by {delta} "
f"(expected {baseline['unconnected']}, got {connectivity['unconnected_pins']})"
)
if "nets_min" in baseline:
net_count = connectivity["net_count"]
if net_count < baseline["nets_min"]:
regressions.append(
f"net count {net_count} below minimum {baseline['nets_min']}"
)
# --- Status ---
status = "pass"
if fatal:
status = "fail"
if regressions:
status = "fail"
result: dict[str, Any] = {
"success": True,
"status": status,
"schematic_path": root_path,
"erc": {
"total_violations": len(violations),
"by_type": by_type,
"by_severity": by_severity,
"fatal": fatal,
"engine": "kicad-cli",
},
"connectivity": connectivity,
}
if baseline_delta is not None:
result["connectivity"]["baseline_delta"] = baseline_delta
if regressions:
result["regressions"] = regressions
# Offload full violation list if large
if len(violations) > INLINE_RESULT_THRESHOLD:
detail_path = write_detail_file(
root_path, "validate_violations.json", violations,
)
result["erc"]["detail_file"] = detail_path
else:
result["erc"]["violations"] = violations
return result

View File

@ -537,3 +537,241 @@ class TestVerifyConnectivity:
for r in result["results"]: for r in result["results"]:
assert "net" in r assert "net" in r
assert "status" in r assert "status" in r
@pytest.mark.unit
class TestValidateSchematic:
"""Tests for the validate_schematic aggregate tool."""
def _make_project(self, tmp_path, violations=None):
"""Create a minimal project structure with mockable ERC."""
pro = tmp_path / "test.kicad_pro"
pro.write_text("{}")
sch = tmp_path / "test.kicad_sch"
sch.write_text('(kicad_sch (version 20231120) (uuid "root"))\n')
if violations is None:
violations = []
erc_json = {
"sheets": [
{
"path": "/",
"uuid_path": "/root",
"violations": violations,
},
],
}
return str(sch), erc_json
def _patch_erc(self, erc_json):
"""Return a combined context manager that mocks kicad-cli ERC."""
import contextlib
import json
from unittest.mock import patch
def fake_subprocess_run(cmd, **kwargs):
out_idx = cmd.index("-o")
out_path = cmd[out_idx + 1]
with open(out_path, "w") as f:
json.dump(erc_json, f)
class FakeResult:
returncode = 0
stdout = ""
stderr = ""
return FakeResult()
stack = contextlib.ExitStack()
stack.enter_context(patch("mckicad.tools.schematic_analysis._HAS_SCH_API", False))
stack.enter_context(patch("mckicad.tools.schematic_analysis.find_kicad_cli", return_value="/usr/bin/kicad-cli"))
stack.enter_context(patch("subprocess.run", side_effect=fake_subprocess_run))
return stack
def test_pass_no_violations(self, tmp_path):
"""Clean schematic returns status=pass."""
from mckicad.tools.schematic_analysis import validate_schematic
sch_path, erc_json = self._make_project(tmp_path)
with self._patch_erc(erc_json):
result = validate_schematic(schematic_path=sch_path)
assert result["success"] is True
assert result["status"] == "pass"
assert result["erc"]["total_violations"] == 0
assert result["erc"]["fatal"] == []
def test_fail_on_fatal_violation_type(self, tmp_path):
"""Violations matching fail_on types cause status=fail."""
from mckicad.tools.schematic_analysis import validate_schematic
violations = [
{
"description": "Net short between VCC and GND",
"severity": "error",
"type": "multiple_net_names",
"items": [],
},
{
"description": "Dangling label",
"severity": "warning",
"type": "global_label_dangling",
"items": [],
},
]
sch_path, erc_json = self._make_project(tmp_path, violations)
with self._patch_erc(erc_json):
result = validate_schematic(schematic_path=sch_path)
assert result["status"] == "fail"
assert result["erc"]["total_violations"] == 2
assert len(result["erc"]["fatal"]) == 1
assert result["erc"]["fatal"][0]["type"] == "multiple_net_names"
assert result["erc"]["by_type"]["multiple_net_names"] == 1
assert result["erc"]["by_type"]["global_label_dangling"] == 1
def test_pass_with_non_fatal_violations(self, tmp_path):
"""Violations not in fail_on list don't cause failure."""
from mckicad.tools.schematic_analysis import validate_schematic
violations = [
{
"description": "Dangling label",
"severity": "warning",
"type": "global_label_dangling",
"items": [],
},
]
sch_path, erc_json = self._make_project(tmp_path, violations)
with self._patch_erc(erc_json):
result = validate_schematic(schematic_path=sch_path)
assert result["status"] == "pass"
assert result["erc"]["total_violations"] == 1
assert result["erc"]["fatal"] == []
def test_custom_fail_on(self, tmp_path):
"""Custom fail_on types are respected."""
from mckicad.tools.schematic_analysis import validate_schematic
violations = [
{
"description": "Wire overlap",
"severity": "error",
"type": "label_multiple_wires",
"items": [],
},
]
sch_path, erc_json = self._make_project(tmp_path, violations)
with self._patch_erc(erc_json):
result = validate_schematic(
schematic_path=sch_path,
fail_on=["label_multiple_wires", "multiple_net_names"],
)
assert result["status"] == "fail"
assert len(result["erc"]["fatal"]) == 1
def test_baseline_pass(self, tmp_path):
"""Baseline comparison passes when metrics match."""
from mckicad.tools.schematic_analysis import validate_schematic
sch_path, erc_json = self._make_project(tmp_path)
with self._patch_erc(erc_json):
result = validate_schematic(
schematic_path=sch_path,
baseline={"connections": 0, "unconnected": 0},
)
assert result["status"] == "pass"
assert result["connectivity"]["baseline_delta"]["connections"] == 0
assert result["connectivity"]["baseline_delta"]["unconnected"] == 0
assert "regressions" not in result
def test_baseline_regression_connections(self, tmp_path):
"""Decreased connections cause regression + fail."""
from mckicad.tools.schematic_analysis import validate_schematic
sch_path, erc_json = self._make_project(tmp_path)
with self._patch_erc(erc_json):
result = validate_schematic(
schematic_path=sch_path,
baseline={"connections": 100},
)
# connectivity returns 0 connections (no sch-api in mock), so delta is -100
assert result["status"] == "fail"
assert result["connectivity"]["baseline_delta"]["connections"] == -100
assert len(result["regressions"]) >= 1
assert "connections decreased" in result["regressions"][0]
def test_baseline_regression_unconnected(self, tmp_path):
"""Increased unconnected pins cause regression + fail."""
from mckicad.tools.schematic_analysis import validate_schematic
sch_path, erc_json = self._make_project(tmp_path)
with self._patch_erc(erc_json):
result = validate_schematic(
schematic_path=sch_path,
baseline={"unconnected": -1},
)
# unconnected_pins is 0, baseline is -1, so delta is +1 (regression)
assert result["status"] == "fail"
assert "regressions" in result
def test_by_severity_counts(self, tmp_path):
"""Severity counts are aggregated correctly."""
from mckicad.tools.schematic_analysis import validate_schematic
violations = [
{"description": "E1", "severity": "error", "type": "t1", "items": []},
{"description": "E2", "severity": "error", "type": "t2", "items": []},
{"description": "W1", "severity": "warning", "type": "t3", "items": []},
]
sch_path, erc_json = self._make_project(tmp_path, violations)
with self._patch_erc(erc_json):
result = validate_schematic(
schematic_path=sch_path,
fail_on=[], # don't fail on any type
)
assert result["status"] == "pass"
assert result["erc"]["by_severity"]["error"] == 2
assert result["erc"]["by_severity"]["warning"] == 1
def test_invalid_path(self):
"""Invalid schematic path returns error."""
from mckicad.tools.schematic_analysis import validate_schematic
result = validate_schematic(schematic_path="/tmp/nonexistent.kicad_sch")
assert result["success"] is False
def test_result_structure(self, tmp_path):
"""Return dict has all expected top-level keys."""
from mckicad.tools.schematic_analysis import validate_schematic
sch_path, erc_json = self._make_project(tmp_path)
with self._patch_erc(erc_json):
result = validate_schematic(schematic_path=sch_path)
assert "success" in result
assert "status" in result
assert "erc" in result
assert "connectivity" in result
assert "total_violations" in result["erc"]
assert "by_type" in result["erc"]
assert "by_severity" in result["erc"]
assert "fatal" in result["erc"]
assert "net_count" in result["connectivity"]
assert "connection_count" in result["connectivity"]
assert "unconnected_pins" in result["connectivity"]