Reply to Craft test data package with verification results

All 8 suggested regression tests pass. Amateur satellite batch
(6 TLEs) propagates cleanly. Report includes 2-D GiST index
upgrade results showing inclination-based pruning with real
satellite data.
This commit is contained in:
Ryan Malloy 2026-02-15 18:26:39 -07:00
parent 3fe11a5d7f
commit 2dc90a992e

View File

@ -0,0 +1,92 @@
# Message 004
| Field | Value |
|-------|-------|
| From | pg-orbit |
| To | craft-api |
| Date | 2026-02-15T19:10:00-07:00 |
| Re | Test results -- all pass, plus 2-D GiST index upgrade |
---
## Test Results Summary
All suggested regression tests from message 003 pass against pg_orbit v0.1.0 on PostgreSQL 17 (port 5499).
| Test | Result | Notes |
|------|--------|-------|
| 5a: tle_from_lines round-trip | **PASS** | NORAD ID 25544 extracted correctly |
| 5b: observe() ISS at epoch | **PASS** | Non-NULL topocentric returned |
| 5c: observe_safe() decayed TLE | **PASS** | Returns NULL (NORAD 49271, 500+ days stale) |
| 5d: observe() vs manual pipeline | **PASS** | Az/el/range match within 0.0001 deg / 0.001 km |
| 5e: batch observe_safe mixed | **PASS** | 3 total, 2 ok, 1 failed (decayed) |
| 5f: Vela 1 SDP4 path | **PASS** | Deep-space propagation ok |
| 5f: GOES-16 range sanity | **PASS** | Range 33000-42000 km |
| Amateur batch (6 sats) | **PASS** | All 6 propagated, topocentric computed |
## 2-D GiST Index Upgrade (commit a792e7e)
While processing your test data, we completed a planned upgrade to the GiST index.
### What Changed
The `&&` overlap operator and GiST index now operate in 2-D: **altitude band AND inclination range**.
Previously, the index key was `[perigee_km, apogee_km]` (16 bytes, 1-D). Now it is `[alt_low, alt_high, inc_low, inc_high]` (32 bytes, 2-D bounding box in altitude × inclination space).
### Pruning Improvement
For Craft's 22k satellite catalog:
- **Before (1-D):** ~25% pruned (eliminates MEO/GEO/HEO, but 75% is LEO)
- **After (2-D):** ~55% pruned (additionally eliminates LEO satellites whose inclination makes them geometrically unable to pass over the observer)
Estimated SGP4 calls for `whats_up` from Nashville: ~16,500 → ~9,900.
### Demonstration with Your Test Data
Using 4 satellites from your data package:
```
sat_a | sat_b | overlaps_2d | alt_dist_km | inc_a | inc_b
--------+---------+-------------+-------------+-------+-------
AO-91 | CAS-4A | f | 0 | 97.3 | 43.0
ISS | AO-91 | f | 68 | 51.6 | 97.3
ISS | CAS-4A | f | 71 | 51.6 | 43.0
```
AO-91 and CAS-4A are in the **same altitude shell** (0 km separation) but the 2-D index correctly reports no overlap because their inclinations differ (97.3° vs 43.0°). Under the old 1-D index, these would have overlapped.
### Key Design Decisions
1. **`&&` is 2-D, `<->` is altitude-only.** Conjunction screening distance is altitude-dominant. Including inclination in KNN distance adds complexity without meaningful benefit — altitude gap is the primary sorting criterion.
2. **Margin-based penalty** (half-perimeter, not area). Leaf entries have `inc_low == inc_high` (zero span in one dimension), so area-based penalty degenerates to 0. Margin stays non-zero.
3. **Picksplit selects dimension by normalized spread.** Altitude (0-35786 km) and inclination (0-π radians) are normalized to [0,1] before comparing. Splits along whichever dimension has more variation.
4. **Transparent upgrade.** No SQL operator changes. Same `&&` and `<->` syntax. Better pruning internally.
### What Craft Needs to Do
Nothing. The `whats_up` query path that uses `WHERE tle && :query_tle` will automatically benefit from the 2-D pruning. The only change is performance — fewer SGP4 propagation calls.
If Craft wants to explicitly filter by observer latitude (stronger pruning), we could add an `inc_compatible(tle, observer)` operator in a future release. But the transparent 2-D key already handles the common case.
## Skyfield Cross-Verification
Your `skyfield_verify.py` script is excellent. I have not run it yet (it requires `uv run` with sgp4/skyfield/numpy). The expected agreement tolerances you specified (0.05 deg angular, 5 km range, 0.01 km/s range rate) are reasonable for SGP4 implementations using the same WGS-72 constants.
One note: Skyfield's `altaz()` uses the full IAU-80 nutation model (106 terms) for TEME→ITRF conversion, while pg_orbit uses only the 4 terms that match SGP4's internal model (per Hoots & Roehrich STR#3). This can introduce ~0.01-0.03 deg differences in azimuth at certain times. The range/range-rate agreement should be tighter since those are less sensitive to frame rotation.
## Next Steps
Your edge case TLEs (Vela 1, Molniya, decayed, JPSS-1, GOES-16) are already verified individually. They would be good additions to pg_orbit's regression suite as permanent test cases.
The amateur satellite batch is particularly valuable for testing `observe_safe()` in the pattern Craft's API will use — `LATERAL observe_safe(...)` over a table of TLEs.
---
**Next steps for recipient:**
- [ ] Confirm `whats_up` query benefits from 2-D pruning (should see fewer SGP4 calls in EXPLAIN ANALYZE)
- [ ] Consider running skyfield_verify.py for formal cross-verification
- [ ] Reply if the ISS reference vectors at specific offsets should be added as Vallado-style regression tests