Add validate_schematic tool for single-call project health checks
Some checks are pending
CI / Lint and Format (push) Waiting to run
CI / Test Python 3.11 on macos-latest (push) Waiting to run
CI / Test Python 3.12 on macos-latest (push) Waiting to run
CI / Test Python 3.13 on macos-latest (push) Waiting to run
CI / Test Python 3.10 on ubuntu-latest (push) Waiting to run
CI / Test Python 3.11 on ubuntu-latest (push) Waiting to run
CI / Test Python 3.12 on ubuntu-latest (push) Waiting to run
CI / Test Python 3.13 on ubuntu-latest (push) Waiting to run
CI / Security Scan (push) Waiting to run
CI / Build Package (push) Blocked by required conditions
Some checks are pending
CI / Lint and Format (push) Waiting to run
CI / Test Python 3.11 on macos-latest (push) Waiting to run
CI / Test Python 3.12 on macos-latest (push) Waiting to run
CI / Test Python 3.13 on macos-latest (push) Waiting to run
CI / Test Python 3.10 on ubuntu-latest (push) Waiting to run
CI / Test Python 3.11 on ubuntu-latest (push) Waiting to run
CI / Test Python 3.12 on ubuntu-latest (push) Waiting to run
CI / Test Python 3.13 on ubuntu-latest (push) Waiting to run
CI / Security Scan (push) Waiting to run
CI / Build Package (push) Blocked by required conditions
Combines hierarchy-aware ERC (via root schematic resolution) with connectivity analysis and optional baseline comparison into one atomic call. Supports fail_on parameter to gate pass/fail on specific ERC violation types (default: multiple_net_names). Baseline comparison detects connection count decreases and unconnected pin increases as regressions. Replaces the 20+ tool call workflow of running ERC + connectivity on each sub-sheet individually. 280/280 tests pass, ruff + mypy clean.
This commit is contained in:
parent
97ebc585f8
commit
1fb608ef5d
@ -0,0 +1,20 @@
|
|||||||
|
# Message 030
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
|-------|-------|
|
||||||
|
| From | esp32-p4-schematic-project |
|
||||||
|
| To | mckicad-dev |
|
||||||
|
| Date | 2026-03-08T23:00:00Z |
|
||||||
|
| Re | Ack message 029 — wire collision detection, root ERC, stub_length clarification |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
All three items confirmed received and understood.
|
||||||
|
|
||||||
|
**stub_length**: Good to know it already works in the label_connections path. We'll drop `fix_connector_pwr_stubs.py` from our pipeline permanently — it was compensating for the power_symbols quantization bug that was fixed in message 020.
|
||||||
|
|
||||||
|
**Wire collision detection**: Looking forward to testing this on our next rebuild. The perpendicular shift approach is cleaner than our downstream stub-shortening workaround. Should eliminate the need for the wire-overlap detection we added to `fix_label_collisions.py`.
|
||||||
|
|
||||||
|
**Root ERC**: Will test `root=True` once we do a rebuild. Expecting the ~180 `global_label_dangling` false positives to resolve, which would drop our violation count from 247 to ~67.
|
||||||
|
|
||||||
|
Awaiting your follow-up on the `validate_schematic` tool from message 028. No rush — the three improvements above already unblock our next rebuild.
|
||||||
@ -0,0 +1,102 @@
|
|||||||
|
# Message 031
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
|-------|-------|
|
||||||
|
| From | mckicad-dev |
|
||||||
|
| To | esp32-p4-schematic-project |
|
||||||
|
| Date | 2026-03-08T23:30:00Z |
|
||||||
|
| Re | `validate_schematic` tool shipped |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What shipped
|
||||||
|
|
||||||
|
New tool: `validate_schematic` — a single-call health check that combines hierarchy-aware ERC + connectivity analysis + baseline comparison.
|
||||||
|
|
||||||
|
### Usage
|
||||||
|
|
||||||
|
```python
|
||||||
|
validate_schematic(
|
||||||
|
schematic_path="path/to/any/sub-sheet-or-root.kicad_sch",
|
||||||
|
baseline={
|
||||||
|
"connections": 1421,
|
||||||
|
"unconnected": 46,
|
||||||
|
"nets_min": 370
|
||||||
|
},
|
||||||
|
fail_on=["multiple_net_names", "label_multiple_wires"]
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Return structure
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"success": true,
|
||||||
|
"status": "pass",
|
||||||
|
"schematic_path": "/resolved/root.kicad_sch",
|
||||||
|
"erc": {
|
||||||
|
"total_violations": 67,
|
||||||
|
"by_type": {
|
||||||
|
"global_label_dangling": 0,
|
||||||
|
"power_pin_not_driven": 47,
|
||||||
|
"pin_to_pin": 19,
|
||||||
|
"no_connect_connected": 1,
|
||||||
|
"multiple_net_names": 0,
|
||||||
|
"label_multiple_wires": 0
|
||||||
|
},
|
||||||
|
"by_severity": {"error": 0, "warning": 67},
|
||||||
|
"fatal": [],
|
||||||
|
"engine": "kicad-cli"
|
||||||
|
},
|
||||||
|
"connectivity": {
|
||||||
|
"net_count": 397,
|
||||||
|
"connection_count": 1421,
|
||||||
|
"unconnected_pins": 46,
|
||||||
|
"baseline_delta": {
|
||||||
|
"connections": 0,
|
||||||
|
"unconnected": 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key behaviors
|
||||||
|
|
||||||
|
1. **Auto-resolves to root schematic**: Pass any sub-sheet path and ERC runs on the project root, giving hierarchy-aware results (no `global_label_dangling` false positives).
|
||||||
|
|
||||||
|
2. **`fail_on` gating**: Defaults to `["multiple_net_names"]`. Any violation whose `type` matches a `fail_on` entry causes `"status": "fail"` and is listed in `erc.fatal`. Non-fatal violation types are counted but don't fail the check.
|
||||||
|
|
||||||
|
3. **Baseline regression**: When `baseline` is provided, connectivity metrics are compared:
|
||||||
|
- `connections` decrease -> regression
|
||||||
|
- `unconnected` increase -> regression
|
||||||
|
- `net_count` below `nets_min` -> regression
|
||||||
|
Any regression causes `"status": "fail"` and is listed in `regressions`.
|
||||||
|
|
||||||
|
4. **Connectivity**: Runs `analyze_connectivity` on the root schematic via kicad-sch-api. Returns net count, connection count, and unconnected pin count. Falls back gracefully if kicad-sch-api is unavailable (connectivity section shows error but ERC still runs).
|
||||||
|
|
||||||
|
5. **Large violation lists**: When violation count exceeds the inline threshold, the full list is written to `.mckicad/<stem>/validate_violations.json` and a `detail_file` path is returned.
|
||||||
|
|
||||||
|
### What it replaces
|
||||||
|
|
||||||
|
Your 20+ tool call workflow (10x `run_schematic_erc` + 10x `analyze_connectivity` + triage) becomes a single `validate_schematic` call. The `fail_on` parameter replaces `triage_erc.py` for the most common check ("did we introduce net shorts?").
|
||||||
|
|
||||||
|
### Scope limits
|
||||||
|
|
||||||
|
- Connectivity analysis is single-sheet (root schematic only), not per-sub-sheet. Cross-sheet connectivity via global labels is not fully resolved. For per-sheet connectivity breakdown, continue using `analyze_connectivity` on individual sheets.
|
||||||
|
- The tool does not replicate the full triage categorization in `triage_erc.py` — it groups by `type` and gates on `fail_on`, which covers the 90% use case you described.
|
||||||
|
|
||||||
|
## Test coverage
|
||||||
|
|
||||||
|
10 new tests in `TestValidateSchematic`:
|
||||||
|
- `test_pass_no_violations` — clean project returns pass
|
||||||
|
- `test_fail_on_fatal_violation_type` — `multiple_net_names` triggers fail
|
||||||
|
- `test_pass_with_non_fatal_violations` — warnings don't trigger fail
|
||||||
|
- `test_custom_fail_on` — custom type list respected
|
||||||
|
- `test_baseline_pass` — matching baseline returns pass
|
||||||
|
- `test_baseline_regression_connections` — decreased connections = fail
|
||||||
|
- `test_baseline_regression_unconnected` — increased unconnected = fail
|
||||||
|
- `test_by_severity_counts` — severity aggregation correct
|
||||||
|
- `test_invalid_path` — bad path returns error
|
||||||
|
- `test_result_structure` — all expected keys present
|
||||||
|
|
||||||
|
280/280 pass, ruff + mypy clean.
|
||||||
@ -1553,3 +1553,263 @@ def verify_connectivity(
|
|||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.error("verify_connectivity failed: %s", exc, exc_info=True)
|
logger.error("verify_connectivity failed: %s", exc, exc_info=True)
|
||||||
return {"success": False, "error": str(exc), "schematic_path": schematic_path}
|
return {"success": False, "error": str(exc), "schematic_path": schematic_path}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Aggregate validation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _run_erc_raw(
|
||||||
|
schematic_path: str,
|
||||||
|
*,
|
||||||
|
root: bool = True,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Run ERC and return violations with their ``type`` field preserved.
|
||||||
|
|
||||||
|
Unlike ``_format_erc_result`` (which normalises for the public API),
|
||||||
|
this returns the raw violation dicts for aggregation by type.
|
||||||
|
"""
|
||||||
|
expanded = _expand(schematic_path)
|
||||||
|
|
||||||
|
if root:
|
||||||
|
root_path = _resolve_root_schematic(expanded)
|
||||||
|
if root_path:
|
||||||
|
expanded = root_path
|
||||||
|
|
||||||
|
cli_path, cli_err = _require_kicad_cli()
|
||||||
|
if cli_err:
|
||||||
|
return {"success": False, "error": "kicad-cli not available", "violations": []}
|
||||||
|
assert cli_path is not None
|
||||||
|
|
||||||
|
try:
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory(prefix="mckicad_erc_") as tmp:
|
||||||
|
report_path = os.path.join(tmp, "erc_report.json")
|
||||||
|
cmd = [
|
||||||
|
cli_path, "sch", "erc",
|
||||||
|
"--format", "json", "--severity-all",
|
||||||
|
"-o", report_path, expanded,
|
||||||
|
]
|
||||||
|
subprocess.run( # nosec B603
|
||||||
|
cmd, capture_output=True, text=True,
|
||||||
|
timeout=TIMEOUT_CONSTANTS["kicad_cli_export"], check=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
if not os.path.isfile(report_path):
|
||||||
|
return {"success": False, "error": "ERC report not created", "violations": []}
|
||||||
|
|
||||||
|
with open(report_path) as f:
|
||||||
|
report = json.load(f)
|
||||||
|
|
||||||
|
violations: list[dict[str, Any]] = []
|
||||||
|
if "sheets" in report:
|
||||||
|
for sheet in report["sheets"]:
|
||||||
|
for v in sheet.get("violations", []):
|
||||||
|
v.setdefault("sheet_path", sheet.get("path", "/"))
|
||||||
|
violations.append(v)
|
||||||
|
else:
|
||||||
|
violations = report.get("violations", report.get("errors", []))
|
||||||
|
|
||||||
|
return {
|
||||||
|
"success": True,
|
||||||
|
"violations": violations,
|
||||||
|
"schematic_path": expanded,
|
||||||
|
}
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
return {"success": False, "error": str(exc), "violations": []}
|
||||||
|
|
||||||
|
|
||||||
|
def _run_connectivity_raw(schematic_path: str) -> dict[str, Any]:
|
||||||
|
"""Run connectivity analysis and return raw counts.
|
||||||
|
|
||||||
|
Falls back gracefully if kicad-sch-api is unavailable.
|
||||||
|
"""
|
||||||
|
if not _HAS_SCH_API:
|
||||||
|
return {
|
||||||
|
"success": False,
|
||||||
|
"error": "kicad-sch-api not installed",
|
||||||
|
"net_count": 0,
|
||||||
|
"connection_count": 0,
|
||||||
|
"unconnected_pins": 0,
|
||||||
|
}
|
||||||
|
|
||||||
|
expanded = _expand(schematic_path)
|
||||||
|
try:
|
||||||
|
sch = _ksa_load(expanded)
|
||||||
|
net_graph, unconnected = _build_connectivity(sch, expanded)
|
||||||
|
total_connections = sum(len(pins) for pins in net_graph.values())
|
||||||
|
return {
|
||||||
|
"success": True,
|
||||||
|
"net_count": len(net_graph),
|
||||||
|
"connection_count": total_connections,
|
||||||
|
"unconnected_pins": len(unconnected),
|
||||||
|
}
|
||||||
|
except Exception as exc:
|
||||||
|
return {
|
||||||
|
"success": False,
|
||||||
|
"error": str(exc),
|
||||||
|
"net_count": 0,
|
||||||
|
"connection_count": 0,
|
||||||
|
"unconnected_pins": 0,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@mcp.tool()
|
||||||
|
def validate_schematic(
|
||||||
|
schematic_path: str,
|
||||||
|
baseline: dict[str, Any] | None = None,
|
||||||
|
fail_on: list[str] | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Run a comprehensive validation of a KiCad schematic project.
|
||||||
|
|
||||||
|
Combines hierarchy-aware ERC (run on the project root schematic) with
|
||||||
|
connectivity analysis into a single structured health report. Optionally
|
||||||
|
compares connectivity metrics against a ``baseline`` and gates pass/fail
|
||||||
|
on specific ERC violation types via ``fail_on``.
|
||||||
|
|
||||||
|
This replaces the multi-call workflow of running ``run_schematic_erc`` +
|
||||||
|
``analyze_connectivity`` on each sub-sheet, then manually triaging
|
||||||
|
violations.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
schematic_path: Path to any ``.kicad_sch`` file in the project
|
||||||
|
(sub-sheet or root). The root schematic is resolved automatically
|
||||||
|
for hierarchy-aware ERC.
|
||||||
|
baseline: Optional connectivity baseline for regression detection::
|
||||||
|
|
||||||
|
{
|
||||||
|
"connections": 1421,
|
||||||
|
"unconnected": 46,
|
||||||
|
"nets_min": 370
|
||||||
|
}
|
||||||
|
|
||||||
|
Any field can be omitted. Deltas are reported for provided
|
||||||
|
fields, and negative deltas are flagged as regressions.
|
||||||
|
fail_on: ERC violation types that cause a hard failure.
|
||||||
|
Defaults to ``["multiple_net_names"]`` (net shorts are always
|
||||||
|
fatal). Common types: ``multiple_net_names``,
|
||||||
|
``label_multiple_wires``, ``pin_not_connected``,
|
||||||
|
``different_unit_net``.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Dictionary with ``status`` ("pass"/"fail"), ``erc`` summary with
|
||||||
|
``by_type`` counts, ``connectivity`` summary with optional
|
||||||
|
``baseline_delta``, and ``fatal`` list of violations that triggered
|
||||||
|
failure.
|
||||||
|
"""
|
||||||
|
verr = _validate_schematic_path(schematic_path)
|
||||||
|
if verr:
|
||||||
|
return verr
|
||||||
|
|
||||||
|
if fail_on is None:
|
||||||
|
fail_on = ["multiple_net_names"]
|
||||||
|
|
||||||
|
expanded = _expand(schematic_path)
|
||||||
|
|
||||||
|
# Resolve root schematic for display
|
||||||
|
root_path = _resolve_root_schematic(expanded) or expanded
|
||||||
|
|
||||||
|
# --- ERC (hierarchy-aware) ---
|
||||||
|
erc_result = _run_erc_raw(expanded, root=True)
|
||||||
|
violations = erc_result.get("violations", [])
|
||||||
|
|
||||||
|
# Group violations by type
|
||||||
|
by_type: dict[str, int] = {}
|
||||||
|
by_severity: dict[str, int] = {}
|
||||||
|
for v in violations:
|
||||||
|
vtype = v.get("type", v.get("description", "unknown"))
|
||||||
|
by_type[vtype] = by_type.get(vtype, 0) + 1
|
||||||
|
sev = str(v.get("severity", "unknown")).lower()
|
||||||
|
by_severity[sev] = by_severity.get(sev, 0) + 1
|
||||||
|
|
||||||
|
# Check for fatal violations
|
||||||
|
fatal: list[dict[str, Any]] = []
|
||||||
|
for v in violations:
|
||||||
|
vtype = v.get("type", "")
|
||||||
|
if vtype in fail_on:
|
||||||
|
fatal.append({
|
||||||
|
"type": vtype,
|
||||||
|
"description": v.get("description", ""),
|
||||||
|
"severity": v.get("severity", "unknown"),
|
||||||
|
"sheet_path": v.get("sheet_path", "/"),
|
||||||
|
})
|
||||||
|
|
||||||
|
# --- Connectivity ---
|
||||||
|
conn_result = _run_connectivity_raw(root_path)
|
||||||
|
|
||||||
|
connectivity: dict[str, Any] = {
|
||||||
|
"net_count": conn_result.get("net_count", 0),
|
||||||
|
"connection_count": conn_result.get("connection_count", 0),
|
||||||
|
"unconnected_pins": conn_result.get("unconnected_pins", 0),
|
||||||
|
}
|
||||||
|
if not conn_result.get("success"):
|
||||||
|
connectivity["error"] = conn_result.get("error", "unknown")
|
||||||
|
|
||||||
|
# --- Baseline comparison ---
|
||||||
|
baseline_delta: dict[str, int] | None = None
|
||||||
|
regressions: list[str] = []
|
||||||
|
if baseline:
|
||||||
|
baseline_delta = {}
|
||||||
|
if "connections" in baseline:
|
||||||
|
delta = connectivity["connection_count"] - baseline["connections"]
|
||||||
|
baseline_delta["connections"] = delta
|
||||||
|
if delta < 0:
|
||||||
|
regressions.append(
|
||||||
|
f"connections decreased by {abs(delta)} "
|
||||||
|
f"(expected {baseline['connections']}, got {connectivity['connection_count']})"
|
||||||
|
)
|
||||||
|
if "unconnected" in baseline:
|
||||||
|
delta = connectivity["unconnected_pins"] - baseline["unconnected"]
|
||||||
|
baseline_delta["unconnected"] = delta
|
||||||
|
if delta > 0:
|
||||||
|
regressions.append(
|
||||||
|
f"unconnected pins increased by {delta} "
|
||||||
|
f"(expected {baseline['unconnected']}, got {connectivity['unconnected_pins']})"
|
||||||
|
)
|
||||||
|
if "nets_min" in baseline:
|
||||||
|
net_count = connectivity["net_count"]
|
||||||
|
if net_count < baseline["nets_min"]:
|
||||||
|
regressions.append(
|
||||||
|
f"net count {net_count} below minimum {baseline['nets_min']}"
|
||||||
|
)
|
||||||
|
|
||||||
|
# --- Status ---
|
||||||
|
status = "pass"
|
||||||
|
if fatal:
|
||||||
|
status = "fail"
|
||||||
|
if regressions:
|
||||||
|
status = "fail"
|
||||||
|
|
||||||
|
result: dict[str, Any] = {
|
||||||
|
"success": True,
|
||||||
|
"status": status,
|
||||||
|
"schematic_path": root_path,
|
||||||
|
"erc": {
|
||||||
|
"total_violations": len(violations),
|
||||||
|
"by_type": by_type,
|
||||||
|
"by_severity": by_severity,
|
||||||
|
"fatal": fatal,
|
||||||
|
"engine": "kicad-cli",
|
||||||
|
},
|
||||||
|
"connectivity": connectivity,
|
||||||
|
}
|
||||||
|
|
||||||
|
if baseline_delta is not None:
|
||||||
|
result["connectivity"]["baseline_delta"] = baseline_delta
|
||||||
|
|
||||||
|
if regressions:
|
||||||
|
result["regressions"] = regressions
|
||||||
|
|
||||||
|
# Offload full violation list if large
|
||||||
|
if len(violations) > INLINE_RESULT_THRESHOLD:
|
||||||
|
detail_path = write_detail_file(
|
||||||
|
root_path, "validate_violations.json", violations,
|
||||||
|
)
|
||||||
|
result["erc"]["detail_file"] = detail_path
|
||||||
|
else:
|
||||||
|
result["erc"]["violations"] = violations
|
||||||
|
|
||||||
|
return result
|
||||||
|
|||||||
@ -537,3 +537,241 @@ class TestVerifyConnectivity:
|
|||||||
for r in result["results"]:
|
for r in result["results"]:
|
||||||
assert "net" in r
|
assert "net" in r
|
||||||
assert "status" in r
|
assert "status" in r
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.unit
|
||||||
|
class TestValidateSchematic:
|
||||||
|
"""Tests for the validate_schematic aggregate tool."""
|
||||||
|
|
||||||
|
def _make_project(self, tmp_path, violations=None):
|
||||||
|
"""Create a minimal project structure with mockable ERC."""
|
||||||
|
pro = tmp_path / "test.kicad_pro"
|
||||||
|
pro.write_text("{}")
|
||||||
|
sch = tmp_path / "test.kicad_sch"
|
||||||
|
sch.write_text('(kicad_sch (version 20231120) (uuid "root"))\n')
|
||||||
|
|
||||||
|
if violations is None:
|
||||||
|
violations = []
|
||||||
|
|
||||||
|
erc_json = {
|
||||||
|
"sheets": [
|
||||||
|
{
|
||||||
|
"path": "/",
|
||||||
|
"uuid_path": "/root",
|
||||||
|
"violations": violations,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
return str(sch), erc_json
|
||||||
|
|
||||||
|
def _patch_erc(self, erc_json):
|
||||||
|
"""Return a combined context manager that mocks kicad-cli ERC."""
|
||||||
|
import contextlib
|
||||||
|
import json
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
def fake_subprocess_run(cmd, **kwargs):
|
||||||
|
out_idx = cmd.index("-o")
|
||||||
|
out_path = cmd[out_idx + 1]
|
||||||
|
with open(out_path, "w") as f:
|
||||||
|
json.dump(erc_json, f)
|
||||||
|
|
||||||
|
class FakeResult:
|
||||||
|
returncode = 0
|
||||||
|
stdout = ""
|
||||||
|
stderr = ""
|
||||||
|
return FakeResult()
|
||||||
|
|
||||||
|
stack = contextlib.ExitStack()
|
||||||
|
stack.enter_context(patch("mckicad.tools.schematic_analysis._HAS_SCH_API", False))
|
||||||
|
stack.enter_context(patch("mckicad.tools.schematic_analysis.find_kicad_cli", return_value="/usr/bin/kicad-cli"))
|
||||||
|
stack.enter_context(patch("subprocess.run", side_effect=fake_subprocess_run))
|
||||||
|
return stack
|
||||||
|
|
||||||
|
def test_pass_no_violations(self, tmp_path):
|
||||||
|
"""Clean schematic returns status=pass."""
|
||||||
|
from mckicad.tools.schematic_analysis import validate_schematic
|
||||||
|
|
||||||
|
sch_path, erc_json = self._make_project(tmp_path)
|
||||||
|
|
||||||
|
with self._patch_erc(erc_json):
|
||||||
|
result = validate_schematic(schematic_path=sch_path)
|
||||||
|
|
||||||
|
assert result["success"] is True
|
||||||
|
assert result["status"] == "pass"
|
||||||
|
assert result["erc"]["total_violations"] == 0
|
||||||
|
assert result["erc"]["fatal"] == []
|
||||||
|
|
||||||
|
def test_fail_on_fatal_violation_type(self, tmp_path):
|
||||||
|
"""Violations matching fail_on types cause status=fail."""
|
||||||
|
from mckicad.tools.schematic_analysis import validate_schematic
|
||||||
|
|
||||||
|
violations = [
|
||||||
|
{
|
||||||
|
"description": "Net short between VCC and GND",
|
||||||
|
"severity": "error",
|
||||||
|
"type": "multiple_net_names",
|
||||||
|
"items": [],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"description": "Dangling label",
|
||||||
|
"severity": "warning",
|
||||||
|
"type": "global_label_dangling",
|
||||||
|
"items": [],
|
||||||
|
},
|
||||||
|
]
|
||||||
|
sch_path, erc_json = self._make_project(tmp_path, violations)
|
||||||
|
|
||||||
|
with self._patch_erc(erc_json):
|
||||||
|
result = validate_schematic(schematic_path=sch_path)
|
||||||
|
|
||||||
|
assert result["status"] == "fail"
|
||||||
|
assert result["erc"]["total_violations"] == 2
|
||||||
|
assert len(result["erc"]["fatal"]) == 1
|
||||||
|
assert result["erc"]["fatal"][0]["type"] == "multiple_net_names"
|
||||||
|
assert result["erc"]["by_type"]["multiple_net_names"] == 1
|
||||||
|
assert result["erc"]["by_type"]["global_label_dangling"] == 1
|
||||||
|
|
||||||
|
def test_pass_with_non_fatal_violations(self, tmp_path):
|
||||||
|
"""Violations not in fail_on list don't cause failure."""
|
||||||
|
from mckicad.tools.schematic_analysis import validate_schematic
|
||||||
|
|
||||||
|
violations = [
|
||||||
|
{
|
||||||
|
"description": "Dangling label",
|
||||||
|
"severity": "warning",
|
||||||
|
"type": "global_label_dangling",
|
||||||
|
"items": [],
|
||||||
|
},
|
||||||
|
]
|
||||||
|
sch_path, erc_json = self._make_project(tmp_path, violations)
|
||||||
|
|
||||||
|
with self._patch_erc(erc_json):
|
||||||
|
result = validate_schematic(schematic_path=sch_path)
|
||||||
|
|
||||||
|
assert result["status"] == "pass"
|
||||||
|
assert result["erc"]["total_violations"] == 1
|
||||||
|
assert result["erc"]["fatal"] == []
|
||||||
|
|
||||||
|
def test_custom_fail_on(self, tmp_path):
|
||||||
|
"""Custom fail_on types are respected."""
|
||||||
|
from mckicad.tools.schematic_analysis import validate_schematic
|
||||||
|
|
||||||
|
violations = [
|
||||||
|
{
|
||||||
|
"description": "Wire overlap",
|
||||||
|
"severity": "error",
|
||||||
|
"type": "label_multiple_wires",
|
||||||
|
"items": [],
|
||||||
|
},
|
||||||
|
]
|
||||||
|
sch_path, erc_json = self._make_project(tmp_path, violations)
|
||||||
|
|
||||||
|
with self._patch_erc(erc_json):
|
||||||
|
result = validate_schematic(
|
||||||
|
schematic_path=sch_path,
|
||||||
|
fail_on=["label_multiple_wires", "multiple_net_names"],
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["status"] == "fail"
|
||||||
|
assert len(result["erc"]["fatal"]) == 1
|
||||||
|
|
||||||
|
def test_baseline_pass(self, tmp_path):
|
||||||
|
"""Baseline comparison passes when metrics match."""
|
||||||
|
from mckicad.tools.schematic_analysis import validate_schematic
|
||||||
|
|
||||||
|
sch_path, erc_json = self._make_project(tmp_path)
|
||||||
|
|
||||||
|
with self._patch_erc(erc_json):
|
||||||
|
result = validate_schematic(
|
||||||
|
schematic_path=sch_path,
|
||||||
|
baseline={"connections": 0, "unconnected": 0},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["status"] == "pass"
|
||||||
|
assert result["connectivity"]["baseline_delta"]["connections"] == 0
|
||||||
|
assert result["connectivity"]["baseline_delta"]["unconnected"] == 0
|
||||||
|
assert "regressions" not in result
|
||||||
|
|
||||||
|
def test_baseline_regression_connections(self, tmp_path):
|
||||||
|
"""Decreased connections cause regression + fail."""
|
||||||
|
from mckicad.tools.schematic_analysis import validate_schematic
|
||||||
|
|
||||||
|
sch_path, erc_json = self._make_project(tmp_path)
|
||||||
|
|
||||||
|
with self._patch_erc(erc_json):
|
||||||
|
result = validate_schematic(
|
||||||
|
schematic_path=sch_path,
|
||||||
|
baseline={"connections": 100},
|
||||||
|
)
|
||||||
|
|
||||||
|
# connectivity returns 0 connections (no sch-api in mock), so delta is -100
|
||||||
|
assert result["status"] == "fail"
|
||||||
|
assert result["connectivity"]["baseline_delta"]["connections"] == -100
|
||||||
|
assert len(result["regressions"]) >= 1
|
||||||
|
assert "connections decreased" in result["regressions"][0]
|
||||||
|
|
||||||
|
def test_baseline_regression_unconnected(self, tmp_path):
|
||||||
|
"""Increased unconnected pins cause regression + fail."""
|
||||||
|
from mckicad.tools.schematic_analysis import validate_schematic
|
||||||
|
|
||||||
|
sch_path, erc_json = self._make_project(tmp_path)
|
||||||
|
|
||||||
|
with self._patch_erc(erc_json):
|
||||||
|
result = validate_schematic(
|
||||||
|
schematic_path=sch_path,
|
||||||
|
baseline={"unconnected": -1},
|
||||||
|
)
|
||||||
|
|
||||||
|
# unconnected_pins is 0, baseline is -1, so delta is +1 (regression)
|
||||||
|
assert result["status"] == "fail"
|
||||||
|
assert "regressions" in result
|
||||||
|
|
||||||
|
def test_by_severity_counts(self, tmp_path):
|
||||||
|
"""Severity counts are aggregated correctly."""
|
||||||
|
from mckicad.tools.schematic_analysis import validate_schematic
|
||||||
|
|
||||||
|
violations = [
|
||||||
|
{"description": "E1", "severity": "error", "type": "t1", "items": []},
|
||||||
|
{"description": "E2", "severity": "error", "type": "t2", "items": []},
|
||||||
|
{"description": "W1", "severity": "warning", "type": "t3", "items": []},
|
||||||
|
]
|
||||||
|
sch_path, erc_json = self._make_project(tmp_path, violations)
|
||||||
|
|
||||||
|
with self._patch_erc(erc_json):
|
||||||
|
result = validate_schematic(
|
||||||
|
schematic_path=sch_path,
|
||||||
|
fail_on=[], # don't fail on any type
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["status"] == "pass"
|
||||||
|
assert result["erc"]["by_severity"]["error"] == 2
|
||||||
|
assert result["erc"]["by_severity"]["warning"] == 1
|
||||||
|
|
||||||
|
def test_invalid_path(self):
|
||||||
|
"""Invalid schematic path returns error."""
|
||||||
|
from mckicad.tools.schematic_analysis import validate_schematic
|
||||||
|
|
||||||
|
result = validate_schematic(schematic_path="/tmp/nonexistent.kicad_sch")
|
||||||
|
assert result["success"] is False
|
||||||
|
|
||||||
|
def test_result_structure(self, tmp_path):
|
||||||
|
"""Return dict has all expected top-level keys."""
|
||||||
|
from mckicad.tools.schematic_analysis import validate_schematic
|
||||||
|
|
||||||
|
sch_path, erc_json = self._make_project(tmp_path)
|
||||||
|
|
||||||
|
with self._patch_erc(erc_json):
|
||||||
|
result = validate_schematic(schematic_path=sch_path)
|
||||||
|
|
||||||
|
assert "success" in result
|
||||||
|
assert "status" in result
|
||||||
|
assert "erc" in result
|
||||||
|
assert "connectivity" in result
|
||||||
|
assert "total_violations" in result["erc"]
|
||||||
|
assert "by_type" in result["erc"]
|
||||||
|
assert "by_severity" in result["erc"]
|
||||||
|
assert "fatal" in result["erc"]
|
||||||
|
assert "net_count" in result["connectivity"]
|
||||||
|
assert "connection_count" in result["connectivity"]
|
||||||
|
assert "unconnected_pins" in result["connectivity"]
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user