warehack.ing/informix-db
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix three tuple-framing bugs: BOOLEAN, NCHAR, INT8/SERIAL8 (2026.05.08.2)

Browse Source 
Data corruption affecting every Informix version including 15. Two of the
three silently corrupt columns AFTER the offending one, so the damage
shows up far from its cause.

BOOLEAN: described as UDTFIXED(41) with encoded_length=1, but carries the
full UDT envelope on the wire — [1-byte indicator][4-byte length][data],
six bytes for a one-byte value. We read one byte and left five behind.
Captured payload for (INT, BOOLEAN 't', INT, VARCHAR 'tail'):
  00 01 b2 07 | 00 00 00 00 01 74 | 00 03 64 0e | 04 74 61 69 6c
Before: (111111, b'\x00', 1, '\x00\x03d\x0e\x04tail')
After:  (111111, True, 222222, 'tail')

NCHAR: fixed-width and space-padded like CHAR, but we had it grouped with
the byte-length-prefixed types. NCHAR(10) holding 'nch' read 0x6E ('n') as
a 110-byte length. Silent truncation alone, struct.error with any column
following. NVARCHAR really is length-prefixed and is unchanged; there's a
test guarding both sides now.

INT8/SERIAL8: absent from FIXED_WIDTHS, fell through to the unknown-type
path and surfaced as raw bytes. INT8 is not BIGINT — 10 bytes,
sign-magnitude, halves stored high-last:
  bytes 0-1 sign word (0=NULL, 1=pos, 0xFFFF=neg)
  bytes 2-5 LOW 32 bits, bytes 6-9 HIGH 32 bits
+n and -n share magnitude bytes, so a two's-complement read is wrong for
every negative while looking correct for every positive.

Not a version bug. Reported against Informix 12, but the same 28-type
round-trip against 12.10.FC12W1DE and 15.0.1.0.3DE produced byte-identical
wire output. It read as version-specific only because INT8/SERIAL8 dominate
12-era schemas while our fixtures use BIGINT.

251 tests missed all three because no fixture used these types. Added
tests/test_type_framing.py (20 integration) and tests/test_int8_unit.py
(14 unit, wire vectors from both servers). Each type is tested twice —
alone, and with trailing columns, since only the latter catches desync.

Verified: 271/271 integration on 15; 241/241 non-smart-LOB on 12.10 (the
LOB tests need an sbspace that image lacks); 20/20 framing tests on both.

Also corrected README and _fastpath docstring, which asserted 12.10
compatibility that had never been tested. The claims held up, but they
were guesses when written and cost a user debugging time.
This commit is contained in:
Ryan Malloy 2026-08-26 23:40:39 -06:00
parent 2d83ed7b45
commit 87b5b7c354
8 changed files with 482 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

61
CHANGELOG.md
View File

@ -2,6 +2,67 @@
All notable changes to `informix-db`. Versioning is [CalVer](https://calver.org/) — `YYYY.MM.DD` for date-based releases, `YYYY.MM.DD.N` for same-day post-releases per PEP 440.
## 2026.05.08.2 — Fix three tuple-framing bugs (BOOLEAN, NCHAR, INT8/SERIAL8)
Data-corruption fixes. All three affect **every** Informix version including 15, and two of them silently corrupt columns *after* the offending one. If your schema uses `BOOLEAN`, `NCHAR`, `INT8`, or `SERIAL8`, upgrade.
Reported from the field against Informix 12. The version turned out to be a red herring — see "On the version question" below.
### BOOLEAN corrupted every following column
The server describes `BOOLEAN` as `UDTFIXED` (41) with `encoded_length = 1`, but `encoded_length` is the size of the *value*, not the field. On the wire it carries the standard UDT envelope — `[1-byte null indicator][4-byte length][data]` — six bytes for a one-byte value. We consumed one byte and left five behind, shifting everything downstream.
Captured payload for `(INT, BOOLEAN 't', INT, VARCHAR 'tail')`:
```
00 01 b2 07 | 00 00 00 00 01 74 | 00 03 64 0e | 04 74 61 69 6c
a=111111 | b — 6 bytes | c=222222 | d = [len 4]"tail"
```
Before: `(111111, b'\x00', 1, '\x00\x03d\x0e\x04tail')`. After: `(111111, True, 222222, 'tail')`.
### NCHAR ate its first character, then crashed
`NCHAR` is fixed-width and space-padded, exactly like `CHAR`. We had it grouped with the byte-length-prefixed types, so `NCHAR(10)` holding `'nch'` read `0x6E` (`'n'`) as a 110-byte length and advanced the offset by 110. Single-column selects silently returned `'ch'`; anything with a following column raised `struct.error`.
`NVARCHAR` *is* byte-length-prefixed and is unchanged — there's now a regression test guarding both sides of that distinction.
### INT8 / SERIAL8 had no decoder
They were absent from `FIXED_WIDTHS` and fell through to the unknown-type path, surfacing as raw `bytes`. No desync (the width happened to match `encoded_length`), just a wrong type.
`INT8` is **not** `BIGINT`. It's 10 bytes, sign-magnitude, with the halves stored high-last:
```
bytes 0-1 sign word: 0 = NULL, 1 = positive, 0xFFFF = negative
bytes 2-5 LOW 32 bits, big-endian unsigned
bytes 6-9 HIGH 32 bits, big-endian unsigned
```
`+n` and `-n` have identical magnitude bytes, so decoding this as a two's-complement int64 is wrong for every negative value while looking fine for every positive one.
### On the version question
The report arrived as "Informix 12 mangles result sets." It isn't a version issue. Running the same 28-type round-trip against **12.10.FC12W1DE** and **15.0.1.0.3DE** produced **byte-identical** wire output — same type codes, same encoded lengths, same values. Both servers were equally broken, and are now equally fixed.
The reason it read as version-specific: `INT8`/`SERIAL8` dominate Informix 12-era schemas (`BIGINT` only arrived in 11.50), while our test fixtures use `BIGINT`. The bugs were always there; older schemas just walk into them far more often.
### Why 251 tests missed this
No fixture used `BOOLEAN`, `NCHAR`, `INT8`, or `SERIAL8`. That's the whole explanation. Coverage of *code paths* was good; coverage of the *type matrix* had holes, and the holes were exactly where the bugs lived.
Added `tests/test_type_framing.py` (20 integration tests) and `tests/test_int8_unit.py` (14 unit tests, wire vectors captured from both servers). Every affected type is now tested twice — once for its own value, once with trailing columns, because the trailing-column case is what catches desync. A single-column test passes while the driver corrupts every real query.
### Verified
- 251/251 integration tests on Informix 15.0.1.0.3DE
- 241/241 non-smart-LOB integration tests on Informix 12.10.FC12W1DE (the LOB tests need an sbspace that image doesn't have configured)
- New framing tests: 20/20 on both servers
### Also
`README.md` and `_fastpath.py` claimed 12.10 compatibility that had never been tested. The claims happened to be correct, but they were guesses when written and cost a user debugging time. Both now state what was measured, on which image, on what date.
## 2026.05.05.12 — Phase 39: Connection-scoped read-ahead buffer
Closes the C-vs-Python bulk-fetch gap to within ~7-15% of IfxPy. The lever was the buffer/I/O machinery, not the codec — Phase 37/38 had already brought the codec to within ~25% of IfxPy's C path; the remaining gap was 450k+ ``read_exact`` calls per 100k-row fetch, each doing its own ``recv``-loop and ``bytes.join``.

10
README.md
View File

@ -149,7 +149,15 @@ The fast-path RPC (`SQ_FPROUTINE` / `SQ_EXFPROUTINE`) bypasses PREPARE → EXECU
## Server compatibility
Tested against IBM Informix Dynamic Server **15.0.1.0.3DE** (the official `icr.io/informix/informix-developer-database` Docker image). The wire protocol is stable across modern Informix versions; should work against 12.10+ unmodified.
| Server | Status |
|---|---|
| **15.0.1.0.3DE** | Primary target. Full suite green (251 integration tests). |
| **12.10.FC12W1DE** | Verified 2026-05-08. Full suite green except smart-LOB, which needs an sbspace we haven't configured on that image — not a driver limitation. |
| **14.10** | Not yet tested. Expected to work; we'll confirm rather than assume. |
Earlier releases of this README claimed the wire protocol was "stable across modern versions" and should "work against 12.10+ unmodified." That was never tested when written. It has since been measured: running the same 28-type round-trip against 12.10 and 15 produces **byte-identical** wire output, so the claim turned out to be true — but it was a guess at the time, and a user lost debugging time to it. Apologies.
One real caveat remains. SQLI negotiates some wire framing through a capability exchange (`SQ_PROTOCOLS`) that this driver does not yet perform; we hardcode the modern framing. That is correct for every server we have measured, but it is an assumption rather than a negotiation, and it is the first thing to suspect if you hit corrupt rows on a server we haven't listed above. Please open an issue with your server version if so.
For features that need server-side configuration (smart-LOBs, logged transactions), see [`docs/DECISION_LOG.md`](https://git.supported.systems/warehack.ing/informix-db/src/branch/main/docs/DECISION_LOG.md):
- Phase 7 — logged-DB transactions

2
pyproject.toml
View File

@ -1,6 +1,6 @@
[project]
name = "informix-driver"
version = "2026.05.08.1"
version = "2026.05.08.2"
description = "Pure-Python driver for IBM Informix IDS — speaks the SQLI wire protocol over raw sockets. No CSDK, no JVM, no native libraries."
readme = "README.md"
license = { text = "MIT" }

15
src/informix_db/_fastpath.py
View File

@ -39,9 +39,18 @@ def build_get_routine_pdu(signature: str) -> bytes:
``[short SQ_GETROUTINE=101][byte isRoutineById=0][int sigLen]
[sig bytes][pad if odd][short fparamFlag=0][short SQ_EOT=12]``
JDBC's ``getJavaToIfxCharBytes`` uses 4-byte length prefix on
modern servers (``isRemove64KLimitSupported``). We always emit the
4-byte form — works against 12.10+ unequivocally.
JDBC's ``getJavaToIfxCharBytes`` uses a 4-byte length prefix when
``isRemove64KLimitSupported()`` (capability bit 62) is set, and a
2-byte prefix otherwise. We always emit the 4-byte form.
Verified correct on 15.0.1.0.3DE and 12.10.FC12W1DE (2026-05-08) —
the fast-path RPC tests pass on both. Not yet checked on 14.10.
An earlier version of this docstring asserted 12.10+ compatibility
"unequivocally" before anyone had run it against 12.10. The claim
happened to hold, but it was a guess. The bit is negotiated via
``SQ_PROTOCOLS``, which this driver does not yet send, so this
remains an assumption on any server we haven't measured.
"""
sig_bytes = signature.encode("iso-8859-1")
sig_len = len(sig_bytes)

61
src/informix_db/_resultset.py
View File

@ -226,6 +226,17 @@ _LENGTH_PREFIXED_SHORT_TYPES = frozenset({
_TC_NVCHAR,
})
# CHAR and NCHAR are **fixed-width**, space-padded to ``encoded_length``.
# VARCHAR and NVARCHAR are byte-length-prefixed. Getting NCHAR wrong is not
# a cosmetic bug: treating it as length-prefixed consumes its first
# character as a length byte, then advances the offset by that value —
# e.g. NCHAR(10) holding 'nch' reads 0x6E ('n') as a 110-byte length and
# desyncs the rest of the row, usually into a struct.error crash.
# Verified on the wire against 12.10 and 15:
# NCHAR(10) 'nch' -> 6e 63 68 20 20 20 20 20 20 20 (10 B, padded)
# NVARCHAR(20) 'nvc' -> 03 6e 76 63 (1-byte prefix)
_FIXED_WIDTH_CHAR_TYPES = frozenset({_TC_CHAR, _TC_NCHAR})
_COMPOSITE_UDT_TYPES = frozenset({
_TC_ROW,
_TC_COLLECTION,
@ -293,12 +304,13 @@ def compile_column_readers(columns: list[ColumnInfo]) -> list[tuple]:
readers.append((_RK_FIXED, FIXED_WIDTHS[tc], DECODERS[tc]))
continue
if tc == _TC_CHAR:
if tc in _FIXED_WIDTH_CHAR_TYPES:
# CHAR and NCHAR: fixed width, space-padded to encoded_length.
readers.append((_RK_CHAR, col.encoded_length, DECODERS[tc]))
continue
if tc in _LENGTH_PREFIXED_SHORT_TYPES:
# VARCHAR / NCHAR / NVCHAR — CHAR was already excluded above.
# VARCHAR / NVARCHAR — CHAR and NCHAR excluded above.
readers.append((_RK_BYTE_PREFIX, DECODERS[tc]))
continue
@ -582,6 +594,28 @@ def _legacy_dispatch_one_column(
cls = BlobLocator if col.extended_id == 10 else ClobLocator
return offset, cls(raw=bytes(raw))
# BOOLEAN. The server describes it as UDTFIXED (41) with
# extended_name='boolean' and encoded_length=1, but ``encoded_length``
# is the size of the *value*, not the field: on the wire it carries the
# standard UDT envelope ``[1-byte null indicator][4-byte length][data]``
# — 6 bytes total for a 1-byte value. Consuming only ``encoded_length``
# leaves 5 bytes on the wire and desyncs every subsequent column.
# Verified payload (INT, BOOLEAN 't', INT, VARCHAR 'tail'):
# 00 01 b2 07 | 00 00 00 00 01 74 | 00 03 64 0e | 04 74 61 69 6c
# The value byte is 0x74 ('t'), which _decode_bool already understands.
if tc == _TC_UDTFIXED and (
col.extended_name == "boolean" or col.extended_id == 5
):
indicator = payload[offset]
offset += 1
length = int.from_bytes(payload[offset:offset + 4], "big", signed=True)
offset += 4
raw = payload[offset:offset + length]
offset += length
if indicator == 1:
return offset, None
return offset, bool(raw and raw[0] in (ord("t"), ord("T"), 1))
# ROW / COLLECTION composite UDT
if tc in _COMPOSITE_UDT_TYPES:
indicator = payload[offset]
@ -785,8 +819,8 @@ def parse_tuple_payload(
# docs/CAPTURES/13-py-varchar.socat.log:
# payload = 09 73 79 73 74 61 62 6c 65 73
# = [byte 9]["systables"]
# CHAR is fixed-width per encoded_length — handled below.
if tc == _TC_CHAR:
# CHAR and NCHAR are fixed-width per encoded_length.
if tc in _FIXED_WIDTH_CHAR_TYPES:
width = col.encoded_length
raw = payload[offset:offset + width]
offset += width
@ -862,6 +896,25 @@ def parse_tuple_payload(
values.append(cls(raw=bytes(raw)))
continue
# BOOLEAN — UDT envelope, not a bare byte. See the matching branch
# in _legacy_dispatch_one_column for the wire evidence.
if tc == _TC_UDTFIXED and (
col.extended_name == "boolean" or col.extended_id == 5
):
indicator = payload[offset]
offset += 1
length = int.from_bytes(
payload[offset:offset + 4], "big", signed=True
)
offset += 4
raw = payload[offset:offset + length]
offset += length
if indicator == 1:
values.append(None)
else:
values.append(bool(raw and raw[0] in (ord("t"), ord("T"), 1)))
continue
# ROW / COLLECTION (Phase 12): composite UDTs. Wire format is
# ``[byte ind][int length][bytes]`` — same shape as
# UDTVAR(lvarchar) above, but the payload semantics are a

44
src/informix_db/converters.py
View File

@ -227,6 +227,41 @@ def _decode_bool(raw: bytes) -> bool:
return raw[0] in (ord("t"), ord("T"), 1)
def _decode_int8(raw: bytes) -> int | None:
"""INT8 / SERIAL8 — the *legacy* 64-bit integer, 10 bytes on the wire.
NOT the same as BIGINT (52) / BIGSERIAL (53), which are plain 8-byte
big-endian. The INT8 layout mirrors ``ifx_int8_t`` and splits the
magnitude across two 32-bit halves in the opposite order you'd guess:
bytes 0-1 sign word: 0 = NULL, 1 = positive, -1 (0xFFFF) = negative
bytes 2-5 LOW 32 bits (unsigned, big-endian)
bytes 6-9 HIGH 32 bits (unsigned, big-endian)
Verified against Informix 12.10.FC12W1DE and 15.0.1.0.3DE, which emit
byte-identical encodings::
123456789012 -> 00 01 | be 99 1a 14 | 00 00 00 1c
-123456789012 -> ff ff | be 99 1a 14 | 00 00 00 1c
42 -> 00 01 | 00 00 00 2a | 00 00 00 00
NULL -> 00 00 | 00 00 00 00 | 00 00 00 00
Note the magnitude bytes are identical for +n and -n — the sign lives
entirely in the leading word, so this is sign-magnitude, not two's
complement. Decoding it as a signed 64-bit integer gives the wrong
answer for every negative value.
"""
if len(raw) < 10:
raise ValueError(f"INT8 payload too short: {len(raw)} bytes, need 10")
sign = _UNPACK_SHORT(raw[0:2])[0]
if sign == 0:
return None
low = int.from_bytes(raw[2:6], "big", signed=False)
high = int.from_bytes(raw[6:10], "big", signed=False)
value = (high << 32) | low
return -value if sign < 0 else value
def _decode_date(raw: bytes) -> datetime.date | None:
"""4-byte big-endian signed int = day count from 1899-12-31. NULL = 0x80000000."""
days = _UNPACK_INT(raw)[0]
@ -534,6 +569,13 @@ FIXED_WIDTHS: dict[int, int] = {
IfxType.BIGSERIAL: 8,
IfxType.DATE: 4,
IfxType.BOOL: 1,
# INT8/SERIAL8 are fixed-width at 10 bytes — NOT 8. See _decode_int8.
# Omitting these was a silent-corruption bug: they fell through to the
# unknown-type path, which surfaces ``encoded_length`` raw bytes.
# ``encoded_length`` happens to be 10 for INT8, so the stream stayed
# aligned and the only symptom was a bytes object where an int belonged.
IfxType.INT8: 10,
IfxType.SERIAL8: 10,
}
@ -557,6 +599,8 @@ DECODERS: dict[int, DecoderFn] = {
IfxType.SERIAL: _decode_int,
IfxType.BIGINT: _decode_bigint,
IfxType.BIGSERIAL: _decode_bigint,
IfxType.INT8: _decode_int8,
IfxType.SERIAL8: _decode_int8,
IfxType.SMFLOAT: _decode_smfloat,
IfxType.FLOAT: _decode_float,
IfxType.CHAR: _decode_char,

71
tests/test_int8_unit.py Normal file
View File

@ -0,0 +1,71 @@
"""Unit tests for the INT8 / SERIAL8 codec — no server required.
The byte vectors below were captured off the wire from both Informix
12.10.FC12W1DE and 15.0.1.0.3DE, which emit byte-identical encodings.
INT8 is sign-magnitude across two 32-bit halves, stored high-half-last:
bytes 0-1 sign word: 0 = NULL, 1 = positive, 0xFFFF = negative
bytes 2-5 LOW 32 bits, big-endian unsigned
bytes 6-9 HIGH 32 bits, big-endian unsigned
The trap: +n and -n have *identical* magnitude bytes. Anything that
treats this as a two's-complement integer is wrong for every negative
value while looking correct for every positive one.
"""
from __future__ import annotations
import pytest
from informix_db._types import IfxType
from informix_db.converters import FIXED_WIDTHS, _decode_int8
# (hex bytes, expected value) — captured from the wire.
WIRE_VECTORS = [
("0001be991a140000001c", 123456789012),
("ffffbe991a140000001c", -123456789012),
("00010000002a00000000", 42),
("ffff0000002a00000000", -42),
("00010000000000000000", 0),
("00000000000000000000", None), # sign word 0 -> NULL
("0001ffffffffffffffff", 2**64 - 1), # both halves saturated
("00010000000100000000", 1),
("00010000000000000001", 1 << 32), # high half only
]
@pytest.mark.parametrize(("hexbytes", "expected"), WIRE_VECTORS)
def test_decode_int8_wire_vectors(hexbytes: str, expected: int | None) -> None:
assert _decode_int8(bytes.fromhex(hexbytes)) == expected
def test_positive_and_negative_share_magnitude_bytes() -> None:
"""The whole reason a naive 8-byte read gets negatives wrong."""
pos = bytes.fromhex("0001be991a140000001c")
neg = bytes.fromhex("ffffbe991a140000001c")
assert pos[2:] == neg[2:], "magnitude bytes should be identical"
assert _decode_int8(pos) == -_decode_int8(neg)
def test_null_sign_word_beats_nonzero_magnitude() -> None:
"""Sign word 0 means NULL regardless of what the magnitude bytes hold."""
assert _decode_int8(bytes.fromhex("0000be991a140000001c")) is None
def test_short_payload_raises() -> None:
with pytest.raises(ValueError, match="too short"):
_decode_int8(bytes.fromhex("0001be991a14")) # 6 bytes, need 10
def test_int8_registered_as_ten_bytes() -> None:
"""A width of 8 here would desync every row containing an INT8."""
assert FIXED_WIDTHS[IfxType.INT8] == 10
assert FIXED_WIDTHS[IfxType.SERIAL8] == 10
def test_int8_width_differs_from_bigint() -> None:
"""INT8 (17) and BIGINT (52) are different types with different
widths — conflating them is the easy mistake."""
assert FIXED_WIDTHS[IfxType.INT8] != FIXED_WIDTHS[IfxType.BIGINT]
assert FIXED_WIDTHS[IfxType.BIGINT] == 8

227
tests/test_type_framing.py Normal file
View File

@ -0,0 +1,227 @@
"""Regression tests for tuple-payload framing bugs found 2026-05-08.
All three bugs below shipped in released versions and survived a
251-test integration suite for one reason: no fixture used the affected
types. They were reported from the field (Informix 12 user) but reproduce
identically on Informix 15 — none of them is version-specific.
Each bug is covered twice: once for the value itself, and once with
**trailing columns** after the affected column. The trailing-column case
is the important one — two of these bugs desynced the row decoder, so
the damage showed up in *later* columns, not the one with the bad type.
A single-column test would have passed while the driver was corrupting
every real query.
Wire evidence for each is recorded in the source comments at the fix
sites (``converters._decode_int8``, ``_resultset._FIXED_WIDTH_CHAR_TYPES``,
and the BOOLEAN branch of ``_resultset._legacy_dispatch_one_column``).
"""
from __future__ import annotations
import pytest
import informix_db
from tests.conftest import ConnParams
pytestmark = pytest.mark.integration
def _connect(conn_params: ConnParams) -> informix_db.Connection:
return informix_db.connect(
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password=conn_params.password,
database=conn_params.database,
server=conn_params.server,
connect_timeout=10.0,
read_timeout=10.0,
)
# --------------------------------------------------------------------------
# INT8 / SERIAL8 — the legacy 64-bit integer. 10 bytes, sign-magnitude,
# with the high and low 32-bit halves stored in the opposite order you'd
# expect. Previously fell through to the unknown-type path and surfaced
# as raw bytes. Correct width, so no desync — just a wrong value.
# --------------------------------------------------------------------------
@pytest.mark.parametrize(
"value",
[
123456789012, # needs both halves
-123456789012, # negative: same magnitude bytes, sign word 0xFFFF
42, # low half only
0,
-1,
2**63 - 1, # INT8 max
-(2**63 - 1),
None, # sign word 0x0000
],
)
def test_int8_round_trip(conn_params: ConnParams, value: int | None) -> None:
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("CREATE TEMP TABLE t_int8 (v INT8)")
cur.execute("INSERT INTO t_int8 VALUES (?)", (value,))
cur.execute("SELECT v FROM t_int8")
assert cur.fetchone() == (value,)
def test_int8_does_not_desync_following_columns(conn_params: ConnParams) -> None:
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute(
"CREATE TEMP TABLE t_int8_mix "
"(a INT, b INT8, c INT, d VARCHAR(10))"
)
cur.execute(
"INSERT INTO t_int8_mix VALUES (?, ?, ?, ?)",
(111111, 123456789012, 222222, "tail"),
)
cur.execute("SELECT a, b, c, d FROM t_int8_mix")
assert cur.fetchone() == (111111, 123456789012, 222222, "tail")
def test_serial8_decodes_as_int(conn_params: ConnParams) -> None:
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("CREATE TEMP TABLE t_ser8 (v SERIAL8, note VARCHAR(8))")
cur.execute("INSERT INTO t_ser8 VALUES (0, 'x')")
cur.execute("SELECT v, note FROM t_ser8")
row = cur.fetchone()
assert row == (1, "x"), f"SERIAL8 row decoded as {row!r}"
def test_int8_is_not_confused_with_bigint(conn_params: ConnParams) -> None:
"""INT8 (17) is 10 bytes sign-magnitude; BIGINT (52) is 8 bytes two's
complement. Same logical range, completely different wire format —
decoding one as the other is silently wrong for negatives."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("CREATE TEMP TABLE t_i8_bi (a INT8, b BIGINT)")
cur.execute("INSERT INTO t_i8_bi VALUES (?, ?)", (-77, -77))
cur.execute("SELECT a, b FROM t_i8_bi")
assert cur.fetchone() == (-77, -77)
# --------------------------------------------------------------------------
# NCHAR — fixed-width and space-padded, exactly like CHAR. Was being read
# as 1-byte-length-prefixed, so the first character got consumed as a
# length and the offset jumped by that value. NCHAR(10) holding 'nch'
# read 0x6E ('n') as a 110-byte length: silent truncation at best, a
# struct.error crash when other columns followed.
# --------------------------------------------------------------------------
def test_nchar_keeps_first_character(conn_params: ConnParams) -> None:
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("CREATE TEMP TABLE t_nchar (v NCHAR(10))")
cur.execute("INSERT INTO t_nchar VALUES ('nch')")
cur.execute("SELECT v FROM t_nchar")
assert cur.fetchone() == ("nch",)
def test_nchar_does_not_desync_following_columns(conn_params: ConnParams) -> None:
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute(
"CREATE TEMP TABLE t_nchar_mix "
"(a INT, n NCHAR(10), c INT, v NVARCHAR(20), z INT)"
)
cur.execute(
"INSERT INTO t_nchar_mix VALUES (111111, 'nch', 222222, 'nvc', 333333)"
)
cur.execute("SELECT a, n, c, v, z FROM t_nchar_mix")
assert cur.fetchone() == (111111, "nch", 222222, "nvc", 333333)
def test_nvarchar_still_length_prefixed(conn_params: ConnParams) -> None:
"""Guard the other side of the NCHAR fix: NVARCHAR *is* byte-length-
prefixed and must not be moved to the fixed-width branch."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("CREATE TEMP TABLE t_nvc (v NVARCHAR(40), tail INT)")
cur.execute("INSERT INTO t_nvc VALUES ('nvc', 4242)")
cur.execute("SELECT v, tail FROM t_nvc")
assert cur.fetchone() == ("nvc", 4242)
# --------------------------------------------------------------------------
# BOOLEAN — described as UDTFIXED(41) with encoded_length=1, but carries
# the full UDT envelope on the wire: [1-byte indicator][4-byte length]
# [data]. Six bytes for a one-byte value. Reading only encoded_length left
# 5 bytes behind and corrupted every subsequent column — this is the bug
# that produced the original field report.
# --------------------------------------------------------------------------
@pytest.mark.parametrize(
("literal", "expected"),
[("'t'", True), ("'f'", False), ("NULL", None)],
)
def test_boolean_decodes(
conn_params: ConnParams, literal: str, expected: bool | None
) -> None:
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("CREATE TEMP TABLE t_bool (v BOOLEAN)")
cur.execute(f"INSERT INTO t_bool VALUES ({literal})")
cur.execute("SELECT v FROM t_bool")
assert cur.fetchone() == (expected,)
def test_boolean_does_not_desync_following_columns(
conn_params: ConnParams,
) -> None:
"""The original field-reported symptom: a BOOLEAN column silently
shifted every column after it by 5 bytes."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute(
"CREATE TEMP TABLE t_bool_mix "
"(a INT, b BOOLEAN, c INT, d VARCHAR(10))"
)
cur.execute("INSERT INTO t_bool_mix VALUES (111111, 't', 222222, 'tail')")
cur.execute("SELECT a, b, c, d FROM t_bool_mix")
assert cur.fetchone() == (111111, True, 222222, "tail")
def test_multiple_booleans_in_one_row(conn_params: ConnParams) -> None:
"""Each BOOLEAN consumes its own 6-byte envelope; an off-by-N in the
envelope compounds across columns."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute(
"CREATE TEMP TABLE t_bool_many "
"(a BOOLEAN, b BOOLEAN, c BOOLEAN, tail INT)"
)
cur.execute("INSERT INTO t_bool_many VALUES ('t', 'f', 't', 5150)")
cur.execute("SELECT a, b, c, tail FROM t_bool_many")
assert cur.fetchone() == (True, False, True, 5150)
# --------------------------------------------------------------------------
# Combined: everything that previously mis-framed, in one row.
# --------------------------------------------------------------------------
def test_all_previously_broken_types_in_one_row(conn_params: ConnParams) -> None:
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute(
"CREATE TEMP TABLE t_framing ("
" a INT, b BOOLEAN, c NCHAR(10), d INT8,"
" e VARCHAR(20), f NVARCHAR(20), g SERIAL8, h INT)"
)
cur.execute(
"INSERT INTO t_framing VALUES "
"(111111, 't', 'nch', 123456789012, 'vc', 'nvc', 0, 999888)"
)
cur.execute("SELECT a, b, c, d, e, f, g, h FROM t_framing")
assert cur.fetchone() == (
111111, True, "nch", 123456789012, "vc", "nvc", 1, 999888,
)