pg_orrery/bench/benchmark_results_66k.txt
Ryan Malloy 747b7ae60a Fix L1 inclination pruning for HEO orbits, add 66k benchmark
Bug: inner_consistent used sma_low for footprint calculation, but
ground footprint grows with altitude. High-SMA bins (GTO, HEO)
need sma_high to compute the maximum footprint — using sma_low
caused 453 false negatives at high-latitude observers (Tromsoe).

Fix: use sma_high (not sma_low) in L1 inclination pruning.

Added regression test: GTO-debris (inc 5 deg, e=0.73) at Tromsoe
must return identical results from seqscan and index scan.

Benchmark on 65,886-object catalog (full Space-Track including
decayed): 80-92% pruning, zero false negatives across 7 query
patterns. SP-GiST beats seqscan for high-latitude observers.
2026-02-17 23:05:49 -07:00

115 lines
4.9 KiB
Plaintext

pg_orrery v0.7.0 SP-GiST Benchmark — 66k Full Space-Track Catalog
===================================================================
Date: 2026-02-17
Catalog: Space-Track USSPACECOM full catalog including decayed (65,886 objects)
Host: Linux 6.16.5-arch1-1, PostgreSQL 17
Branch: phase/spgist-orbital-trie
Note: After fixing L1 inclination pruning (sma_low -> sma_high)
Catalog Composition:
LEO (<128 min): 59,537 (90.4%)
MEO (128-720 min): 3,474 ( 5.3%)
GEO/HEO (720-1500 min): 2,643 ( 4.0%)
Super-GEO (>1500 min): 232 ( 0.4%)
Index Build:
SP-GiST: 55.2 ms, 11 MB
GiST: 118.2 ms, 13 MB
Table: 10 MB
==============================================================
TIMING RESULTS (best of 2-3 runs, ms)
==============================================================
Query Pattern | Seqscan | SP-GiST | Delta
-----------------------------|---------|---------|-------
2h window, Eagle ID, 10 deg | 12.5 | 14.0 | +1.5
6h window, Eagle ID, 10 deg | 12.2 | 15.6 | +3.4
2h window, Tromsø, 10 deg | 11.3 | 10.9 | -0.4 ★
24h window, Eagle ID, 10 deg | 12.0 | 16.2 | +4.2
★ Tromsø (69.6°N): SP-GiST beats seqscan. High-latitude observers
benefit most from inclination pruning.
==============================================================
PRUNING RESULTS
==============================================================
Query Pattern | Candidates | % Pass | % Pruned
-----------------------------|------------|--------|---------
2h window, Eagle ID, 10 deg | 12,964 | 19.7% | 80.3%
6h window, Eagle ID, 10 deg | 24,274 | 36.8% | 63.2%
24h window, Eagle ID, 10 deg | 60,875 | 92.4% | 7.6%
2h window, Eagle ID, 30 deg | 9,680 | 14.7% | 85.3%
2h window, equatorial, 10deg | 9,699 | 14.7% | 85.3%
2h window, Tromsø 69.6°N | 6,529 | 9.9% | 90.1%
2h window, South Pole 85°S | 5,248 | 8.0% | 92.0%
==============================================================
CONSISTENCY CHECKS (all patterns)
==============================================================
Query Pattern | False Negatives | False Positives
-----------------------------|-----------------|----------------
2h Eagle 10deg | 0 | 0
6h Eagle 10deg | 0 | 0
24h Eagle 10deg | 0 | 0
2h Eagle 30deg | 0 | 0
2h Equator 10deg | 0 | 0
2h Tromsø 10deg | 0 | 0
2h South Pole 10deg | 0 | 0
==============================================================
SCALING TREND (2h Eagle 10deg, best-of-N)
==============================================================
Catalog Size | Seqscan | SP-GiST | Delta | Notes
-------------|---------|---------|--------|------
14,376 | 4.5 ms | 6.1 ms | +1.6ms | Active CelesTrak
29,784 | 5.2 ms | 5.2 ms | +0.0ms | Active Space-Track (before fix)
65,886 | 12.5 ms | 14.0 ms | +1.5ms | Full catalog incl decayed (after fix)
The fix (sma_high instead of sma_low for footprint) adds ~1-2ms overhead
by conservatively keeping more subtrees alive during L1 pruning. This is
the correct trade-off: zero false negatives is non-negotiable.
==============================================================
PLANNER BEHAVIOR (66k)
==============================================================
PostgreSQL still chooses SP-GiST Index Only Scan by default:
Index Only Scan using bench_spgist on bench_catalog
Index Cond: (tle &? ...)
Heap Fetches: 0
Buffers: shared hit=4990
Seqscan would read 1,297 pages. Index reads 4,990 pages (3.8x more).
But Index Only Scan avoids all heap I/O.
==============================================================
KEY FINDING: HIGH-LATITUDE OBSERVERS
==============================================================
The SP-GiST index is most valuable for high-latitude observers:
Tromsø (69.6°N): 90.1% pruned, SP-GiST BEATS seqscan by 0.4ms
South Pole (85°S): 92.0% pruned
High-latitude locations eliminate most LEO satellites via the
inclination filter — only satellites with inc > ~60° can reach
these latitudes. The SP-GiST trie prunes entire inclination
subtrees at L1, making the index scan faster than touching
every page in the table.
==============================================================
WHAT THE 80-92% PRUNING MEANS IN PRACTICE
==============================================================
For a 65,886-object catalog with a 2-hour window:
- Without &? operator: 65,886 SGP4 predict_passes() calls
- With &? operator: 12,964 SGP4 calls (Eagle) or 5,248 (South Pole)
- Savings: 52,922-60,638 unnecessary propagation calls avoided
At ~1ms per predict_passes() call (7-day window, 30s resolution),
that's 53-61 seconds of saved computation per query.