Ryan Malloy
e9aed6ce59
Third-pass optimization on parse_tuple_payload's hot loop. Previous phases removed redundant work; this one removes correct-but-wasteful work: the if/elif chain checked branches in implementation order, not frequency order. Fixed-width types (INT, FLOAT, DATE, BIGINT - the most common columns in real queries) sat at the bottom, paying ~7 frozenset misses per column. Changes (src/informix_db/_resultset.py): * Added _FIXED_WIDTH_TYPES = frozenset(FIXED_WIDTHS.keys()) at module load. * New fast-path branch at the TOP of parse_tuple_payload's loop body that handles every _FIXED_WIDTH_TYPES column inline: one frozenset check, one dict lookup, one decode, continue. Skips every other branch. * Cleaned up the bottom fall-through; it now genuinely only catches unknown types. Performance vs Phase 24 baseline: * parse_tuple_5cols_iso8859: 1659 ns -> 1400 ns (-16%) * parse_tuple_5cols_utf8: 1649 ns -> 1341 ns (-19%) Cumulative vs Phase 21 baseline (before any optimization): * parse_tuple_5cols: 2796 ns -> 1400 ns (-50%) - HALF the time * decode_int: 230 ns -> 139 ns (-40%) Margaret Hamilton review surfaced one HIGH finding addressed before tagging: * H: The fast-path optimization assumes every FIXED_WIDTHS key is decodable WITHOUT qualifier inspection (encoded_length etc.). True today, but a future contributor adding a fixed-width type that needs qualifier bits (like DATETIME does) would silently get wrong decode behavior - Lauren-Bug class failure. Fix: added INVARIANT comment to FIXED_WIDTHS in converters.py AND added tests/test_resultset_invariants.py with three CI tripwire tests: - _FIXED_WIDTH_TYPES is disjoint from every other dispatch branch - Every FIXED_WIDTHS key has a DECODERS entry - DECODERS keys stay < 0x100 (Phase 24 collision-free guarantee) The tests carry instructions: if one fires, don't update the test to match - either restore the property or refactor the optimization. Comments rot when nobody reads them; tests fail loudly. baseline.json refreshed; 72 unit + 224 integration + 28 bench = 324 tests; ruff clean.
99 lines
4.0 KiB
Python
99 lines
4.0 KiB
Python
"""Phase 25 — invariant tripwires for parse_tuple_payload's fast-path dispatch.
|
|
|
|
These tests don't exercise behavior. They lock down the structural
|
|
invariants the optimized hot loop in :func:`informix_db._resultset.parse_tuple_payload`
|
|
relies on for correctness. Each test is a CI tripwire — if a future
|
|
contributor breaks an invariant, these fail at test time rather than
|
|
at a customer's wire-protocol mismatch six months later.
|
|
|
|
Lessons from Margaret Hamilton's review of Phases 23/24/25:
|
|
|
|
* The optimization is *correct* — but its correctness depends on
|
|
properties of unrelated tables (DECODERS keys, FIXED_WIDTHS keys,
|
|
IfxType flag bits) staying consistent.
|
|
* A comment at the table only helps if the next contributor reads it.
|
|
* A test fails loudly the moment the invariant is broken. Prefer that.
|
|
|
|
If one of these tests fires, **do not** simply update the test to
|
|
match the new state — that defeats the purpose. Instead read the
|
|
docstring on the failed test and the corresponding INVARIANT comment
|
|
in the source; either restore the property or refactor the
|
|
optimization to no longer depend on it.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from informix_db._resultset import (
|
|
_COMPOSITE_UDT_TYPES,
|
|
_FIXED_WIDTH_TYPES,
|
|
_LENGTH_PREFIXED_SHORT_TYPES,
|
|
_NUMERIC_TYPES,
|
|
_TC_DATETIME,
|
|
_TC_INTERVAL,
|
|
_TC_LVARCHAR,
|
|
_TC_UDTFIXED,
|
|
_TC_UDTVAR,
|
|
)
|
|
from informix_db.converters import DECODERS, FIXED_WIDTHS
|
|
|
|
|
|
def test_fixed_width_types_disjoint_from_other_dispatch_sets() -> None:
|
|
"""parse_tuple_payload's fast path is silently wrong if the FIXED_WIDTHS
|
|
type set overlaps any other branch.
|
|
|
|
The optimization in ``parse_tuple_payload`` puts the FIXED_WIDTHS
|
|
branch FIRST. If a type is also in (e.g.) _NUMERIC_TYPES, the fast
|
|
path swallows it before the DECIMAL/MONEY-specific handler runs —
|
|
silently producing wrong values.
|
|
|
|
If this test fails, you've added a new entry somewhere that
|
|
overlaps. Either move it to FIXED_WIDTHS exclusively (and remove
|
|
its specialized branch) or remove it from FIXED_WIDTHS.
|
|
"""
|
|
other_branch_types = (
|
|
_LENGTH_PREFIXED_SHORT_TYPES
|
|
| _NUMERIC_TYPES
|
|
| _COMPOSITE_UDT_TYPES
|
|
| {_TC_LVARCHAR, _TC_DATETIME, _TC_INTERVAL, _TC_UDTFIXED, _TC_UDTVAR}
|
|
)
|
|
overlap = _FIXED_WIDTH_TYPES & other_branch_types
|
|
assert overlap == set(), (
|
|
f"FIXED_WIDTHS overlap with another parse_tuple_payload branch: {overlap}. "
|
|
f"See the INVARIANT comment on FIXED_WIDTHS in converters.py."
|
|
)
|
|
|
|
|
|
def test_every_fixed_width_type_has_a_decoder() -> None:
|
|
"""The fast path calls ``_decode_base(tc, raw, encoding)`` for every
|
|
FIXED_WIDTHS key. If a key has no entry in DECODERS, we'd raise
|
|
``NotImplementedError`` for that column — surprising the user.
|
|
|
|
If this test fails, you've added a key to FIXED_WIDTHS without
|
|
adding a corresponding decoder. Add the decoder, or remove the
|
|
key.
|
|
"""
|
|
missing = [tc for tc in FIXED_WIDTHS if tc not in DECODERS]
|
|
assert missing == [], (
|
|
f"FIXED_WIDTHS has keys without DECODERS entries: {missing}. "
|
|
f"Every fixed-width type must be decodable by _decode_base."
|
|
)
|
|
|
|
|
|
def test_decoders_keys_stay_below_0x100() -> None:
|
|
"""The Phase 24 optimization in ``_decode_base`` skips ``base_type()``
|
|
by relying on a structural guarantee: all DECODERS keys are ≤ 0xFF
|
|
and all flag bits in _types.py are ≥ 0x100, so a flagged type code
|
|
cannot coincidentally match a DECODERS key.
|
|
|
|
If this test fails, you've added a decoder for a type code with
|
|
bits ≥ 0x100. The collision-free guarantee weakens — re-introduce
|
|
``base_type()`` inside ``_decode_base`` (and remove the Phase 24
|
|
optimization), OR keep the new key but verify it cannot clash with
|
|
any flagged input.
|
|
"""
|
|
high_keys = [tc for tc in DECODERS if tc >= 0x100]
|
|
assert high_keys == [], (
|
|
f"DECODERS contains keys with bits >= 0x100: {high_keys}. "
|
|
f"See the INVARIANT comment on DECODERS in converters.py."
|
|
)
|