warehack.ing/informix-db
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Decode the SQ_PROTOCOLS capability negotiation (2026.08.27)

Browse Source 
Closes the last open item from the Informix 12 field report. The driver
hardcodes several wire-framing choices that SQLI actually negotiates.
Those choices are correct on every server we've measured, but "correct
as far as we know" and "checked" are different things, and a framing
mismatch corrupts rows silently.

We were already sending SQ_PROTOCOLS with the same 8-byte client offer
IBM's JDBC driver uses, and discarding the reply. Now we decode it.

New informix_db.ServerCapabilities, reachable from any connection:

    conn.server_capabilities.four_byte_offset
    conn.server_capabilities.varchar_var_len
    conn.server_capabilities.violated_assumptions()   # [] when we agree
    conn.server_version

violated_assumptions() names each place we emit or parse a fixed wire
shape that is actually capability-gated. Non-empty at connect time logs
a warning naming the bit, so an untested server produces a diagnosable
complaint instead of quiet corruption.

Nothing branches on these bits yet — this is observation and validation
only. The hardcoded framing is correct on all three supported servers,
and rewriting working parse paths to be conditional without a server
that needs it trades certainty for risk.

The measurement, and why 12/14/15 are interchangeable:

  15.0.1.0.3       bdbe9ffe7fb7ffef ff
  14.10.FC7W1      bdbe9ffe7fb7ffef f8
  12.10.FC12W1DE   bdbe9ffe7fb7ffef f0
                   ^^^^^^^^^^^^^^^^ identical

The first 64 bits are byte-identical. Those releases don't merely behave
alike, they negotiate exactly the same capability set.

Two details worth recording. The reply is NINE bytes; JDBC's
enhancedProtocolMechanism switches on case 0..7 and drops the ninth, so
its BitSet(64) never sees it — yet that dropped byte is the only part
that differs between releases. And Cap_1 in the login response is not a
server version, it's the client's declared protocol level echoed back,
which is why JDBC tests == 316 rather than >=. The version string there
is the internal one: 12.10 reports 9.56, 14.10 reports 9.59. At the
protocol level both really are 9.x servers.

This also retires isUSVER as a red herring: it's one of six bits JDBC
pre-sets for any non-zero Cap_1, and Java's BitSet.set never clears, so
it is true on every modern server regardless of the mask.

Separately, fixed __version__. The distribution was renamed informix-db
-> informix-driver on 2026-05-08 but __init__ kept looking up the old
name. importlib.metadata needs the distribution name, not the module
name, and the miss fails silently — so every install of the renamed
package has reported "0.0.0+local". It escaped notice because a stale
informix-db distribution lingered in the dev venv and answered the
query. tests/test_package_metadata.py now pins the name against
[project].name; verified it catches the bug by reintroducing it.

247/247 integration on 15, 14.10, and 12.10. 120 unit tests.
This commit is contained in:
Ryan Malloy 2026-08-27 02:18:08 -06:00
parent 783b2bec97
commit 36478f02c4
9 changed files with 889 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

58
CHANGELOG.md
View File

@ -2,6 +2,64 @@
All notable changes to `informix-db`. Versioning is [CalVer](https://calver.org/) — `YYYY.MM.DD` for date-based releases, `YYYY.MM.DD.N` for same-day post-releases per PEP 440.
## 2026.08.27 — Decode the SQ_PROTOCOLS capability negotiation
Closes the last open item from the Informix 12 field report. The driver hardcodes several wire-framing choices that SQLI actually *negotiates*; those choices were correct on every server we'd measured, but "correct as far as we know" and "checked" are different things, and the failure mode for a framing mismatch is silently corrupted rows.
### What changed
We were already sending `SQ_PROTOCOLS` with the same 8-byte client offer IBM's JDBC driver uses — and throwing the server's reply away. Now we decode it.
New `informix_db.ServerCapabilities`, reachable from any connection:
```python
conn = informix_db.connect(...)
caps = conn.server_capabilities
caps.four_byte_offset # describe uses 4-byte string-table/field-index widths
caps.varchar_var_len # VARCHAR is length-prefixed, not width-padded
caps.remove_64k_limit # 4-byte length prefixes in the fast-path
caps.violated_assumptions() # [] when our hardcoded framing matches
conn.server_version # from the login response
```
`violated_assumptions()` is the point of the exercise. It names each place we emit or parse a fixed wire shape that is actually capability-gated, and returns empty when the server agrees. On connect, a non-empty result logs a warning naming the specific bit. A server we've never tested now produces a diagnosable complaint instead of quiet corruption.
Nothing branches on these bits yet — this release is observation and validation only. That's deliberate: the hardcoded framing is correct on all three supported servers, and swapping working code for newly-written conditional code without a server that needs it would add risk for no benefit.
### The measurement
All three servers negotiate the same thing:
| Server | Negotiated mask |
|---|---|
| 15.0.1.0.3 | `bdbe9ffe7fb7ffef` `ff` |
| 14.10.FC7W1 | `bdbe9ffe7fb7ffef` `f8` |
| 12.10.FC12W1DE | `bdbe9ffe7fb7ffef` `f0` |
**The first eight bytes are byte-identical.** That is the root explanation for everything in the 2026.05.08.2 investigation: 12.10, 14.10, and 15 don't merely behave similarly, they negotiate exactly the same 64-bit capability set. `violated_assumptions()` returns empty on all three.
Two details worth recording:
The reply is **nine** bytes, not eight. JDBC's `enhancedProtocolMechanism` dispatches on `case 0..7` and silently discards the ninth, so its `BitSet(64)` never sees it. We decode it, because it is the *only* part of the mask that differs between releases — newer feature flags that this JDBC build predates.
`Cap_1` in the login response is **not** a server version. It's the client's own declared protocol level echoed back, which is why JDBC tests `== 316` rather than `>=`. The actual version string there is the *internal* one: Informix 12.10 reports `9.56`, 14.10 reports `9.59`. At the protocol level both really are 9.x servers, which is why the marketing version jump didn't move the wire format.
This also retires a red herring. `isUSVER` (bit 2) is one of six bits JDBC pre-sets for any non-zero `Cap_1`, and Java's `BitSet.set` never clears — so it is true on every modern server regardless of the mask. It can't explain version-specific behaviour and isn't worth chasing.
### Also fixed: `__version__` was reporting a sentinel
Found incidentally while bumping the version. The distribution was renamed `informix-db` to `informix-driver` on 2026-05-08, but `__init__.py` kept calling `version("informix-db")`. `importlib.metadata` needs the *distribution* name, not the module name, and the miss fails silently — `__version__` degrades to `"0.0.0+local"` rather than raising.
So every install of the renamed package has been reporting `0.0.0+local`. It escaped notice locally because a stale `informix-db` distribution was still present in the dev environment and answered the lookup with its own old version.
Added `tests/test_package_metadata.py`, which asserts the name passed to `importlib.metadata.version()` matches `[project].name` in `pyproject.toml`. Verified it actually catches the bug by reintroducing it.
### Tests
24 unit tests over the captured masks (no server needed) and 6 integration tests that assert live negotiation. The integration ones run on every server in `make test-matrix`, so the assumption check is re-verified on 12.10, 14.10, and 15 each time.
Full suite: **247 / 247** on all three.
## 2026.05.08.2 — Fix three tuple-framing bugs (BOOLEAN, NCHAR, INT8/SERIAL8)
Data-corruption fixes. All three affect **every** Informix version including 15, and two of them silently corrupt columns *after* the offending one. If your schema uses `BOOLEAN`, `NCHAR`, `INT8`, or `SERIAL8`, upgrade.

15
README.md
View File

@ -172,7 +172,20 @@ Beyond the suite passing, a 28-type round-trip (every scalar type we support, in
Earlier releases of this README claimed the protocol was "stable across modern versions" and should "work against 12.10+ unmodified." That claim turned out to be correct, but it was written before anyone had run it against 12.10, and a user lost debugging time to a version-compatibility problem that didn't exist. It's measured now. Apologies for the earlier guess.
One caveat remains, and it's a real one. SQLI negotiates some wire framing through a capability exchange (`SQ_PROTOCOLS`) that this driver does not yet perform — we hardcode the modern framing. That is correct on all three servers above, but it is an assumption rather than a negotiation. If you hit corrupted rows on a server not in that table, this is the first thing to suspect; please open an issue with your server version and the `cursor.description` of the offending query.
### Capability negotiation
SQLI settles some wire framing through a capability exchange (`SQ_PROTOCOLS`) rather than fixing it by version. This driver decodes that exchange and checks it against the framing it emits:
```python
conn = informix_db.connect(...)
conn.server_version # 'IBM Informix Dynamic Server Version 15.0.1.0.3'
conn.server_capabilities.four_byte_offset # True
conn.server_capabilities.violated_assumptions() # [] — hardcoded framing matches
```
All three servers above negotiate an identical 64-bit capability set, and `violated_assumptions()` is empty on each. If it ever returns something, the driver logs a warning naming the specific bit at connect time — a framing mismatch otherwise shows up as silently corrupted rows, which is a miserable thing to debug.
The framing itself is still hardcoded rather than driven by those bits. That's a deliberate limit: it's correct everywhere we've measured, and rewriting working parse paths to be conditional without a server that needs it would trade certainty for risk. If you hit corrupted rows on a server outside that table, check `violated_assumptions()` first, then please open an issue with your server version, the capability mask, and the `cursor.description` of the offending query.
For features that need server-side configuration (smart-LOBs, logged transactions), see [`docs/DECISION_LOG.md`](https://git.supported.systems/warehack.ing/informix-db/src/branch/main/docs/DECISION_LOG.md):
- Phase 7 — logged-DB transactions

2
pyproject.toml
View File

@ -1,6 +1,6 @@
[project]
name = "informix-driver"
version = "2026.05.08.2"
version = "2026.08.27"
description = "Pure-Python driver for IBM Informix IDS — speaks the SQLI wire protocol over raw sockets. No CSDK, no JVM, no native libraries."
readme = "README.md"
license = { text = "MIT" }

11
src/informix_db/__init__.py
View File

@ -23,6 +23,7 @@ from __future__ import annotations
import ssl
from importlib.metadata import PackageNotFoundError, version
from ._capabilities import ServerCapabilities
from .connections import Connection
from .converters import (
BlobLocator,
@ -56,9 +57,14 @@ threadsafety = 1 # threads may share the module but not connections
paramstyle = "numeric" # locked in DECISION_LOG.md — matches Informix ESQL/C
try:
__version__ = version("informix-db")
# NOTE: the *distribution* is ``informix-driver``; the *module* is
# ``informix_db``. They differ, so this string can't be derived from
# ``__name__``. This looked up the old ``informix-db`` distribution
# until 2026.08.27, which meant __version__ silently reported
# "0.0.0+local" for anyone who installed the renamed package.
__version__ = version("informix-driver")
except PackageNotFoundError:
# Editable install or running uninstalled; fall back to a sentinel.
# Running from a source checkout without an install.
__version__ = "0.0.0+local"
__all__ = [
@ -80,6 +86,7 @@ __all__ = [
"PoolTimeoutError",
"ProgrammingError",
"RowValue",
"ServerCapabilities",
"Warning",
"__version__",
"apilevel",

269
src/informix_db/_capabilities.py Normal file
View File

@ -0,0 +1,269 @@
"""Server capability decoding — the SQ_PROTOCOLS feature bitmap.
SQLI does not have a single "protocol version". It has a 64-bit feature
bitmap that client and server negotiate at connect time, and several bits
in it change **wire framing**, not just feature availability. Get one of
those wrong and the row decoder desyncs.
We already perform the negotiation (``Connection._init_session`` sends
``SQ_PROTOCOLS`` with the same 8-byte client offer the IBM JDBC driver
uses). Until now we discarded the server's reply and hardcoded the modern
framing. This module decodes the reply so the assumptions can at least be
*checked* instead of merely believed.
Two independent sources feed the bitmap, mirroring
``IfxSqliConnect.getServerVer`` / ``enhancedProtocolMechanism``:
1. ``Cap_1`` from the login (SLTYPE_CONACC) response. This is the client's
own protocol level echoed back — 316 means "speaks the enhanced
protocol". It is **not** a server version number, which is why JDBC
tests ``== 316`` rather than ``>=``.
2. The 8-byte mask from the SQ_PROTOCOLS reply.
When ``Cap_1`` is non-zero, JDBC unconditionally pre-sets bits
{0, 2, 3, 4, 49, 51} *before* applying the mask, and a Java ``BitSet.set``
only ever sets — never clears. So those six bits are true on any modern
server regardless of what the mask says. ``isUSVER`` (bit 2) is one of
them, which is why it never varies in practice and is a dead end when
chasing version-specific behaviour.
Bit numbering is MSB-first within each byte: byte 0 bit 0x80 is bit 0,
byte 0 bit 0x01 is bit 7, byte 1 bit 0x80 is bit 8, and so on.
A note on length. The client offer is 8 bytes, but every server we have
tested replies with **nine**. JDBC's ``enhancedProtocolMechanism``
dispatches on ``switch (i) case 0..7`` and silently discards anything
past byte 7, so its ``BitSet(64)`` never sees the ninth byte. We decode
it — bits 64-71 — because it turns out to be the only part of the mask
that differs between releases::
Informix 15.0.1.0.3 bdbe9ffe7fb7ffef ff
Informix 14.10.FC7W1 bdbe9ffe7fb7ffef f8
Informix 12.10.FC12W1DE bdbe9ffe7fb7ffef f0
^^^^^^^^^^^^^^^^ identical
The first 64 bits being byte-identical across those three releases is
why they speak an indistinguishable SQLI dialect. Bits 64+ have no known
meaning here; they are surfaced for diagnostics, not branched on.
"""
from __future__ import annotations
from dataclasses import dataclass, field
# The client's capability offer. Byte-for-byte the value IBM's JDBC driver
# sends (``IfxSqliConnect.clientProtocols``), replayed verbatim — the bits
# are a fixed constant there too, never computed.
CLIENT_PROTOCOLS_MASK = bytes.fromhex("fffc7ffc3c8caa97")
# ``Cap_1`` value meaning "enhanced protocol negotiation applies". JDBC
# compares with equality, not >=, because the server echoes the client's
# own declared level rather than reporting its own.
ENHANCED_PROTOCOL_CAP = 316
# Bits JDBC pre-sets for any non-zero Cap_1, before the mask is applied.
_PRESET_BITS = frozenset({0, 2, 3, 4, 49, 51})
# Named bits. Only the ones we can justify from the decompiled JDBC are
# listed; the mask has 64 and most are irrelevant to us.
BIT_USVER = 2 # isUSVER — always true on modern servers
BIT_LONG_ID_A = 10 # isLongID is bit 10 OR bit 18
BIT_LONG_ID_B = 18
BIT_VARCHAR_VAR_LEN = 13 # gated further by bit 40 + client ifxPADVARCHAR
BIT_DESCRIBE_INPUT = 34
BIT_LVARCHAR_GT_2K = 37
BIT_PAD_VARCHAR_GATE = 40
BIT_FP_DESCRIBE = 45
BIT_AUTO_GENERATED_KEYS = 48
BIT_CUR_SESS_INFO = 49
BIT_FOUR_BYTE_OFFSET = 50 # describe: string-table + field-index widths
BIT_GLS = 51
BIT_NAMED_PARAMETERS = 52
BIT_BIGINT = 54
BIT_SAVEPOINT = 56
BIT_SQ_BATCH = 61
BIT_REMOVE_64K_LIMIT = 62 # 4-byte vs 2-byte length prefixes
BIT_TWO_GB_FETCH_BUFFER = 63
def _decode_bits(mask: bytes) -> set[int]:
"""Expand a big-endian, MSB-first bit mask into a set of bit numbers."""
bits: set[int] = set()
for byte_index, value in enumerate(mask):
for offset in range(8):
if value & (0x80 >> offset):
bits.add(byte_index * 8 + offset)
return bits
@dataclass(frozen=True)
class ServerCapabilities:
"""Decoded SQ_PROTOCOLS feature bitmap plus login-response metadata.
``bits`` is the authoritative set; the named properties are readability
sugar over it. ``raw_mask`` is kept so a bug report can include the
exact bytes without needing a packet capture.
"""
bits: frozenset[int] = field(default_factory=frozenset)
raw_mask: bytes = b""
cap_1: int = 0
cap_2: int = 0
cap_3: int = 0
server_version: str = ""
serial_number: str = ""
applid_name: str = ""
@classmethod
def from_wire(
cls,
mask: bytes,
*,
cap_1: int = 0,
cap_2: int = 0,
cap_3: int = 0,
server_version: str = "",
serial_number: str = "",
applid_name: str = "",
) -> ServerCapabilities:
bits = _decode_bits(mask)
if cap_1 != 0:
# Mirrors JDBC: pre-set bits are OR-ed in and never cleared.
bits |= _PRESET_BITS
return cls(
bits=frozenset(bits),
raw_mask=bytes(mask),
cap_1=cap_1,
cap_2=cap_2,
cap_3=cap_3,
server_version=server_version,
serial_number=serial_number,
applid_name=applid_name,
)
def has(self, bit: int) -> bool:
return bit in self.bits
# -- named capabilities ------------------------------------------------
@property
def usver(self) -> bool:
return self.has(BIT_USVER)
@property
def long_id(self) -> bool:
return self.has(BIT_LONG_ID_A) or self.has(BIT_LONG_ID_B)
@property
def varchar_var_len(self) -> bool:
"""VARCHAR carries a 1-byte length prefix rather than being padded
to its full declared width.
JDBC additionally consults the client-side ``ifxPADVARCHAR``
setting; we never enable padding, so the client half is constant.
"""
return self.has(BIT_VARCHAR_VAR_LEN)
@property
def four_byte_offset(self) -> bool:
"""Describe records use 4-byte string-table size and field indices
rather than 2-byte."""
return self.has(BIT_FOUR_BYTE_OFFSET)
@property
def bigint(self) -> bool:
return self.has(BIT_BIGINT)
@property
def lvarchar_gt_2k(self) -> bool:
return self.has(BIT_LVARCHAR_GT_2K)
@property
def remove_64k_limit(self) -> bool:
"""4-byte length prefixes in the fast-path/SQ_FILE paths."""
return self.has(BIT_REMOVE_64K_LIMIT)
@property
def two_gb_fetch_buffer(self) -> bool:
return self.has(BIT_TWO_GB_FETCH_BUFFER)
@property
def gls(self) -> bool:
return self.has(BIT_GLS)
@property
def describe_input(self) -> bool:
return self.has(BIT_DESCRIBE_INPUT)
@property
def named_parameters(self) -> bool:
return self.has(BIT_NAMED_PARAMETERS)
@property
def savepoint(self) -> bool:
return self.has(BIT_SAVEPOINT)
@property
def enhanced_protocol(self) -> bool:
return self.cap_1 == ENHANCED_PROTOCOL_CAP
# -- assumption checking ----------------------------------------------
def violated_assumptions(self) -> list[str]:
"""Framing assumptions this driver hardcodes that the server
contradicts.
Each entry names a place where we emit or parse a fixed wire shape
that is actually capability-gated. An empty list means every
hardcoded choice matches what the server negotiated.
This is a diagnostic, not a guarantee: it only covers bits we know
to be framing-relevant from the decompiled JDBC. It exists so that
a server we have never tested produces a loud, specific warning
instead of silently corrupted rows.
"""
problems: list[str] = []
if not self.four_byte_offset:
problems.append(
"server did not negotiate 4-byte describe offsets "
f"(bit {BIT_FOUR_BYTE_OFFSET}), but parse_describe reads "
"4-byte string-table size and field indices; column "
"metadata will be misparsed"
)
if not self.varchar_var_len:
problems.append(
f"server did not negotiate variable-length VARCHAR "
f"(bit {BIT_VARCHAR_VAR_LEN}); VARCHAR values are padded to "
"the declared column width on the wire, but the row decoder "
"expects a 1-byte length prefix"
)
if not self.remove_64k_limit:
problems.append(
f"server did not negotiate the 64K limit removal "
f"(bit {BIT_REMOVE_64K_LIMIT}); fast-path routine signatures "
"are emitted with a 4-byte length prefix that the server "
"expects to be 2-byte"
)
return problems
def __repr__(self) -> str:
named = [
name
for name, value in (
("usver", self.usver),
("4byte_offset", self.four_byte_offset),
("varchar_var_len", self.varchar_var_len),
("bigint", self.bigint),
("long_id", self.long_id),
("lvarchar>2k", self.lvarchar_gt_2k),
("remove_64k", self.remove_64k_limit),
("gls", self.gls),
)
if value
]
return (
f"ServerCapabilities(cap_1={self.cap_1}, "
f"mask={self.raw_mask.hex()}, "
f"version={self.server_version!r}, "
f"flags={'|'.join(named) or 'none'})"
)

170
src/informix_db/connections.py
View File

@ -12,6 +12,7 @@ reference in ``docs/CAPTURES/01-connect-only.socat.log``.
from __future__ import annotations
import contextlib
import logging
import os
import socket as socket_mod
import ssl
@ -21,6 +22,7 @@ from io import BytesIO
from pathlib import Path
from . import _auth
from ._capabilities import CLIENT_PROTOCOLS_MASK, ServerCapabilities
from ._messages import (
APPL_ID,
APPL_TYPE,
@ -69,6 +71,9 @@ _LOCALE_ENCODING_MAP = {
}
_log = logging.getLogger(__name__)
def _extract_server_error_text(payload: bytes) -> str | None:
"""Pull the longest printable run out of an opaque rejection payload.
@ -113,6 +118,74 @@ def _python_encoding_from_locale(locale: str) -> str:
return _LOCALE_ENCODING_MAP.get(suffix, "iso-8859-1")
def _decode_conacc(rest: bytes) -> dict | None:
"""Decode the SLTYPE_CONACC server-metadata block.
Mirrors ``com.informix.asf.Connection.DecodeAscBinary``. ``rest`` is
the login response with its 2-byte total-length prefix already
stripped, i.e. starting at the SLType byte.
Layout, verified byte-for-byte against Informix 12.10, 14.10 and 15::
byte SLType (2 = CONACC)
3 skip
short 100 marker
short 101 marker
4 skip
short len; skip len (codeset id, e.g. "IEEEI")
short 108 marker — JDBC hard-fails if this isn't 108
12 skip
short len; bytes server version string
short len; bytes serial number
short len; bytes applid name
int Cap_1 client protocol level, echoed (316 = enhanced)
int Cap_2
int Cap_3
Returns ``None`` if the block doesn't match that shape. Callers must
treat that as "no capability info", never as a connection failure —
every server we support connects fine without any of this being
decodable, and a future server variant should degrade, not break.
"""
try:
offset = 1 + 3 # SLType + 3 skipped bytes
marker_a, marker_b = struct.unpack_from("!hh", rest, offset)
offset += 4
if marker_a != 100 or marker_b != 101:
return None
offset += 4
(codeset_len,) = struct.unpack_from("!h", rest, offset)
offset += 2 + codeset_len
(marker_c,) = struct.unpack_from("!h", rest, offset)
offset += 2
if marker_c != 108:
return None
offset += 12
strings: list[str] = []
for _ in range(3):
(length,) = struct.unpack_from("!h", rest, offset)
offset += 2
if length < 0:
return None
strings.append(
rest[offset : offset + length].rstrip(b"\x00").decode(
"iso-8859-1", "replace"
)
)
offset += length
cap_1, cap_2, cap_3 = struct.unpack_from("!iii", rest, offset)
except (struct.error, IndexError, UnicodeDecodeError):
return None
return {
"server_version": strings[0],
"serial_number": strings[1],
"applid_name": strings[2],
"cap_1": cap_1,
"cap_2": cap_2,
"cap_3": cap_3,
}
# Default environment variables sent in the login PDU (SQ_ASCENV section).
# These match what the JDBC driver sends for a vanilla en_US.8859-1
# connection. Anything missing makes the server fall back to defaults.
@ -205,6 +278,13 @@ class Connection:
# SQ_GETROUTINE; subsequent calls skip that round-trip.
self._fp_handle_cache: dict[str, tuple[str, int]] = {}
# Server metadata from the login response and the SQ_PROTOCOLS
# negotiation. Both are populated during connect; both degrade to
# None rather than failing the connection.
self._conacc: dict | None = None
self._server_protocols: bytes | None = None
self._capabilities: ServerCapabilities | None = None
# Build the env-var dict sent in the login PDU.
self._env = dict(_DEFAULT_ENV)
self._env["CLIENT_LOCALE"] = client_locale
@ -589,9 +669,9 @@ class Connection:
# The 8-byte protocols mask is the JDBC reference value from
# docs/CAPTURES/02-select-1.socat.log; we replay it verbatim
# since the bits are opaque (server-recognized features).
protocols_mask = bytes.fromhex("fffc7ffc3c8caa97")
self._send_protocols(protocols_mask)
self._drain_to_eot()
self._send_protocols(CLIENT_PROTOCOLS_MASK)
self._drain_to_eot() # captures the reply into self._server_protocols
self._build_capabilities()
# Step 2: SQ_INFO with INFO_ENV subtype + session env vars.
# The actual on-wire format (from JDBC's sendEnv at IfxSqli.java
@ -626,6 +706,54 @@ class Connection:
self._send_dbopen(self._database)
self._drain_to_eot()
def _build_capabilities(self) -> None:
"""Assemble ``ServerCapabilities`` from the login response and the
SQ_PROTOCOLS reply, and warn about contradicted assumptions.
This is observation only — nothing in the parse paths branches on
these bits yet. The value is diagnostic: this driver hardcodes
several wire-framing choices that SQLI actually negotiates, and
those choices are correct on every server we have measured
(Informix 12.10, 14.10, 15). On some server we have not measured
they might not be, and the failure mode for a framing mismatch is
silently corrupted rows, which is about the worst way for a
database driver to fail. A log warning naming the specific bit
turns that into something diagnosable from a bug report.
Deliberately never raises: a server that declines to negotiate is
not necessarily broken, and refusing the connection over a
diagnostic would be a worse outcome than proceeding.
"""
if self._server_protocols is None:
return
conacc = self._conacc or {}
try:
caps = ServerCapabilities.from_wire(
self._server_protocols,
cap_1=conacc.get("cap_1", 0),
cap_2=conacc.get("cap_2", 0),
cap_3=conacc.get("cap_3", 0),
server_version=conacc.get("server_version", ""),
serial_number=conacc.get("serial_number", ""),
applid_name=conacc.get("applid_name", ""),
)
except Exception:
_log.debug("could not decode server capabilities", exc_info=True)
return
self._capabilities = caps
problems = caps.violated_assumptions()
if problems:
_log.warning(
"Informix server %r negotiated capabilities that contradict "
"this driver's hardcoded wire framing (mask=%s). Result rows "
"may be decoded incorrectly. Please report this with the "
"mask and your server version. Details: %s",
caps.server_version or "<unknown version>",
caps.raw_mask.hex(),
"; ".join(problems),
)
def _send_protocols(self, protocols: bytes) -> None:
"""Emit a SQ_PROTOCOLS PDU per ``IfxSqli.sendProtocols``.
@ -668,9 +796,13 @@ class Connection:
elif tag == MessageType.SQ_PROTOCOLS:
# ``[short payloadLen][bytes payload][byte 0 if odd-len pad]``
# Then the loop continues and consumes the next tag (usually SQ_EOT).
# The payload is the negotiated 64-bit feature bitmap; stash
# it for _capabilities decoding. Reading it was always
# necessary for stream alignment — we just used to throw it
# away instead of looking at it.
payload_len = struct.unpack("!h", self._sock.read_exact(2))[0]
if payload_len > 0:
self._sock.read_exact(payload_len)
self._server_protocols = self._sock.read_exact(payload_len)
if payload_len & 1:
self._sock.read_exact(1) # writePadded's even-alignment pad
elif tag == MessageType.SQ_DONE:
@ -883,6 +1015,28 @@ class Connection:
# -- response parsing -------------------------------------------------
@property
def server_capabilities(self):
"""The negotiated ``SQ_PROTOCOLS`` feature bitmap, or ``None``.
``None`` means the negotiation reply couldn't be decoded — the
connection still works, we just have no capability information.
See ``informix_db._capabilities.ServerCapabilities``.
"""
return self._capabilities
@property
def server_version(self) -> str:
"""Server version string from the login response.
Note this is the *internal* protocol version, which is not the
marketing version. Informix 12.10 reports ``9.56``, 14.10 reports
``9.59``, and 15 reports ``15.0.1.0.3``. That is why the two older
releases speak an identical SQLI dialect: at the protocol level
they are both 9.x servers.
"""
return self._conacc.get("server_version", "") if self._conacc else ""
def _parse_login_response(self) -> None:
"""Read and parse the server's login response.
@ -916,8 +1070,12 @@ class Connection:
)
elif sl_type != SLHeader.SLTYPE_CONACC:
raise ProtocolError(f"unknown SLType in login response: {sl_type}")
# SLTYPE_CONACC — connection accepted. We don't (yet) decode the
# full server-side metadata. Phase 1 just needs to know "we got in".
# SLTYPE_CONACC — connection accepted. Decode the server metadata
# block for the capability fields. Best-effort: a decode failure
# here must not fail an otherwise-good connection, since every
# server we've tested connects fine without any of this.
self._conacc = _decode_conacc(rest)
def _raise_from_rejection(self, reader: IfxStreamReader) -> None:
"""Best-effort decode of the connection-rejection error block.

98
tests/test_capabilities.py Normal file
View File

@ -0,0 +1,98 @@
"""Integration tests for live SQ_PROTOCOLS negotiation.
These run against whatever server ``IFX_PORT`` points at, so `make
test-matrix` exercises them on 12.10, 14.10, and 15 in turn.
The important one is ``test_no_violated_assumptions``: this driver
hardcodes several wire-framing choices that SQLI actually negotiates, and
a mismatch corrupts rows silently. That test turns "we assume this" into
"we check this on every supported server".
"""
from __future__ import annotations
import pytest
import informix_db
from informix_db._capabilities import ENHANCED_PROTOCOL_CAP, ServerCapabilities
from tests.conftest import ConnParams
pytestmark = pytest.mark.integration
def _connect(conn_params: ConnParams) -> informix_db.Connection:
return informix_db.connect(
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password=conn_params.password,
database=conn_params.database,
server=conn_params.server,
connect_timeout=10.0,
read_timeout=10.0,
)
def test_capabilities_are_decoded(conn_params: ConnParams) -> None:
with _connect(conn_params) as conn:
caps = conn.server_capabilities
assert isinstance(caps, ServerCapabilities)
assert caps.raw_mask, "server sent an empty protocols reply"
assert caps.bits
def test_no_violated_assumptions(conn_params: ConnParams) -> None:
"""Every wire-framing shape we hardcode is one the server negotiated.
If this fails, rows are being decoded against the wrong framing and
the fix is to branch on the capability rather than assume it.
"""
with _connect(conn_params) as conn:
caps = conn.server_capabilities
assert caps is not None
assert caps.violated_assumptions() == []
def test_enhanced_protocol_negotiated(conn_params: ConnParams) -> None:
"""Cap_1 is the client's declared protocol level echoed back. Every
supported server accepts 316."""
with _connect(conn_params) as conn:
caps = conn.server_capabilities
assert caps is not None
assert caps.cap_1 == ENHANCED_PROTOCOL_CAP
assert caps.enhanced_protocol
def test_framing_capabilities_present(conn_params: ConnParams) -> None:
with _connect(conn_params) as conn:
caps = conn.server_capabilities
assert caps is not None
assert caps.four_byte_offset
assert caps.varchar_var_len
assert caps.remove_64k_limit
assert caps.usver
def test_server_version_is_exposed(conn_params: ConnParams) -> None:
"""The version here is the *internal* protocol version, not the
marketing one: 12.10 reports 9.56 and 14.10 reports 9.59. Assert the
shape rather than a value so this holds across the matrix."""
with _connect(conn_params) as conn:
version = conn.server_version
assert "Informix" in version
assert "Version" in version
def test_capabilities_survive_multiple_connections(
conn_params: ConnParams,
) -> None:
"""Negotiation happens per-connection; two connections to the same
server must agree."""
with _connect(conn_params) as first, _connect(conn_params) as second:
assert first.server_capabilities is not None
assert second.server_capabilities is not None
assert (
first.server_capabilities.raw_mask
== second.server_capabilities.raw_mask
)
assert first.server_capabilities.bits == second.server_capabilities.bits

200
tests/test_capabilities_unit.py Normal file
View File

@ -0,0 +1,200 @@
"""Unit tests for SQ_PROTOCOLS capability decoding — no server required.
The masks below were captured from live servers on 2026-08-27. Their most
interesting property is that the first eight bytes are identical across
Informix 12.10, 14.10, and 15 — which is why those three releases speak an
indistinguishable SQLI dialect.
"""
from __future__ import annotations
import dataclasses
import pytest
from informix_db._capabilities import (
BIT_FOUR_BYTE_OFFSET,
BIT_REMOVE_64K_LIMIT,
BIT_USVER,
BIT_VARCHAR_VAR_LEN,
CLIENT_PROTOCOLS_MASK,
ENHANCED_PROTOCOL_CAP,
ServerCapabilities,
_decode_bits,
)
MASK_15 = bytes.fromhex("bdbe9ffe7fb7ffefff")
MASK_1410 = bytes.fromhex("bdbe9ffe7fb7ffeff8")
MASK_1210 = bytes.fromhex("bdbe9ffe7fb7ffeff0")
ALL_MASKS = [
pytest.param(MASK_15, id="15.0.1.0.3"),
pytest.param(MASK_1410, id="14.10.FC7W1"),
pytest.param(MASK_1210, id="12.10.FC12W1DE"),
]
# --------------------------------------------------------------------------
# Bit expansion
# --------------------------------------------------------------------------
def test_bit_numbering_is_msb_first() -> None:
assert _decode_bits(b"\x80") == {0}
assert _decode_bits(b"\x01") == {7}
assert _decode_bits(b"\x00\x80") == {8}
assert _decode_bits(b"\x00\x01") == {15}
def test_bit_expansion_of_first_captured_byte() -> None:
"""0xBD is 1011 1101 -> bits 0, 2, 3, 4, 5, 7."""
assert _decode_bits(b"\xbd") == {0, 2, 3, 4, 5, 7}
def test_empty_mask_yields_no_bits() -> None:
assert _decode_bits(b"") == set()
# --------------------------------------------------------------------------
# The captured masks
# --------------------------------------------------------------------------
def test_first_eight_bytes_identical_across_versions() -> None:
"""The finding this whole module exists to record: 12.10, 14.10, and 15
negotiate exactly the same 64-bit capability set. Only the 9th byte —
which IBM's own JDBC driver discards — differs."""
assert MASK_15[:8] == MASK_1410[:8] == MASK_1210[:8]
assert len({MASK_15[8], MASK_1410[8], MASK_1210[8]}) == 3
@pytest.mark.parametrize("mask", ALL_MASKS)
def test_framing_bits_set_on_every_tested_server(mask: bytes) -> None:
"""Every wire-framing choice this driver hardcodes is one the server
actually negotiated. If this ever fails, the row decoder is wrong."""
caps = ServerCapabilities.from_wire(mask, cap_1=ENHANCED_PROTOCOL_CAP)
assert caps.four_byte_offset, "describe offsets would be misparsed"
assert caps.varchar_var_len, "VARCHAR framing would be misparsed"
assert caps.remove_64k_limit, "fast-path length prefix would be wrong"
assert caps.bigint
assert caps.long_id
assert caps.lvarchar_gt_2k
assert caps.gls
@pytest.mark.parametrize("mask", ALL_MASKS)
def test_no_violated_assumptions_on_tested_servers(mask: bytes) -> None:
caps = ServerCapabilities.from_wire(mask, cap_1=ENHANCED_PROTOCOL_CAP)
assert caps.violated_assumptions() == []
@pytest.mark.parametrize("mask", ALL_MASKS)
def test_ninth_byte_is_decoded(mask: bytes) -> None:
"""JDBC drops bits 64+; we keep them because they're the only part
that varies between releases."""
caps = ServerCapabilities.from_wire(mask, cap_1=ENHANCED_PROTOCOL_CAP)
assert any(b >= 64 for b in caps.bits)
# --------------------------------------------------------------------------
# Pre-set bits — the reason isUSVER is a dead end
# --------------------------------------------------------------------------
def test_preset_bits_applied_when_cap1_nonzero() -> None:
"""JDBC pre-sets {0,2,3,4,49,51} for any non-zero Cap_1 and BitSet.set
never clears, so these are true on any modern server regardless of the
mask. That's why isUSVER never varies and can't explain version-specific
behaviour."""
caps = ServerCapabilities.from_wire(b"\x00" * 9, cap_1=ENHANCED_PROTOCOL_CAP)
assert {0, 2, 3, 4, 49, 51}.issubset(caps.bits)
assert caps.usver
def test_preset_bits_not_applied_when_cap1_zero() -> None:
caps = ServerCapabilities.from_wire(b"\x00" * 9, cap_1=0)
assert caps.bits == frozenset()
assert not caps.usver
def test_preset_cannot_clear_a_bit_the_mask_set() -> None:
caps = ServerCapabilities.from_wire(MASK_15, cap_1=ENHANCED_PROTOCOL_CAP)
assert caps.has(BIT_USVER)
assert caps.has(BIT_FOUR_BYTE_OFFSET)
# --------------------------------------------------------------------------
# Assumption violations — the diagnostic path
# --------------------------------------------------------------------------
def test_missing_four_byte_offset_is_reported() -> None:
# Everything set except bit 50.
bits = bytearray(b"\xff" * 9)
bits[BIT_FOUR_BYTE_OFFSET // 8] &= ~(0x80 >> (BIT_FOUR_BYTE_OFFSET % 8)) & 0xFF
caps = ServerCapabilities.from_wire(bytes(bits), cap_1=ENHANCED_PROTOCOL_CAP)
problems = caps.violated_assumptions()
assert len(problems) == 1
assert "4-byte describe offsets" in problems[0]
def test_missing_varchar_var_len_is_reported() -> None:
bits = bytearray(b"\xff" * 9)
bits[BIT_VARCHAR_VAR_LEN // 8] &= ~(0x80 >> (BIT_VARCHAR_VAR_LEN % 8)) & 0xFF
caps = ServerCapabilities.from_wire(bytes(bits), cap_1=ENHANCED_PROTOCOL_CAP)
problems = caps.violated_assumptions()
assert len(problems) == 1
assert "variable-length VARCHAR" in problems[0]
def test_missing_remove_64k_is_reported() -> None:
bits = bytearray(b"\xff" * 9)
bits[BIT_REMOVE_64K_LIMIT // 8] &= ~(0x80 >> (BIT_REMOVE_64K_LIMIT % 8)) & 0xFF
caps = ServerCapabilities.from_wire(bytes(bits), cap_1=ENHANCED_PROTOCOL_CAP)
problems = caps.violated_assumptions()
assert len(problems) == 1
assert "64K limit" in problems[0]
def test_all_zero_mask_reports_every_assumption() -> None:
caps = ServerCapabilities.from_wire(b"\x00" * 9, cap_1=ENHANCED_PROTOCOL_CAP)
assert len(caps.violated_assumptions()) == 3
# --------------------------------------------------------------------------
# Misc
# --------------------------------------------------------------------------
def test_client_offer_matches_jdbc_reference() -> None:
"""IfxSqliConnect.clientProtocols, verbatim. Changing this changes what
the server negotiates, so it is pinned."""
jdbc_reference = bytes([0xFF, 0xFC, 0x7F, 0xFC, 0x3C, 0x8C, 0xAA, 0x97])
assert jdbc_reference == CLIENT_PROTOCOLS_MASK
assert len(CLIENT_PROTOCOLS_MASK) == 8
def test_enhanced_protocol_requires_exact_cap1() -> None:
"""JDBC tests == 316, not >=, because the server echoes the client's own
declared level rather than reporting its own version."""
assert ServerCapabilities.from_wire(MASK_15, cap_1=316).enhanced_protocol
assert not ServerCapabilities.from_wire(MASK_15, cap_1=315).enhanced_protocol
assert not ServerCapabilities.from_wire(MASK_15, cap_1=317).enhanced_protocol
def test_repr_is_readable_and_includes_mask() -> None:
caps = ServerCapabilities.from_wire(
MASK_15, cap_1=316, server_version="IBM Informix Dynamic Server Version 15"
)
text = repr(caps)
assert "bdbe9ffe7fb7ffefff" in text
assert "usver" in text
assert "cap_1=316" in text
def test_capabilities_are_frozen() -> None:
"""Immutable so a connection's negotiated state can't be edited after
the fact and quietly disagree with what's on the wire."""
caps = ServerCapabilities.from_wire(MASK_15, cap_1=316)
with pytest.raises(dataclasses.FrozenInstanceError):
caps.cap_1 = 1 # type: ignore[misc]

76
tests/test_package_metadata.py Normal file
View File

@ -0,0 +1,76 @@
"""Guards on the package's own metadata.
The distribution is ``informix-driver``; the import module is
``informix_db``. That mismatch is a live trap: ``importlib.metadata``
needs the *distribution* name, so the string can't be derived from
``__name__``, and getting it wrong fails silently — ``__version__``
degrades to the ``0.0.0+local`` sentinel instead of raising.
That is exactly what shipped between the 2026-05-08 rename and
2026.08.27: ``__init__`` still asked for ``informix-db``, so anyone who
installed the renamed package saw ``0.0.0+local``. It went unnoticed
because a stale ``informix-db`` distribution lingered in the dev
environment and answered the query.
"""
from __future__ import annotations
from pathlib import Path
import pytest
import tomllib
import informix_db
PYPROJECT = Path(__file__).resolve().parent.parent / "pyproject.toml"
@pytest.fixture(scope="module")
def pyproject() -> dict:
if not PYPROJECT.is_file():
pytest.skip("pyproject.toml not present (installed-package test run)")
with PYPROJECT.open("rb") as handle:
return tomllib.load(handle)
def test_version_is_not_the_unresolved_sentinel() -> None:
"""If this fails, the distribution name in ``__init__`` no longer
matches ``[project].name`` and every user sees a bogus version."""
assert informix_db.__version__ != "0.0.0+local", (
"__version__ fell back to its sentinel — the distribution name "
"looked up in informix_db/__init__.py does not match the installed "
"package. Check [project].name in pyproject.toml."
)
def test_version_matches_pyproject(pyproject: dict) -> None:
declared = pyproject["project"]["version"]
# PEP 440 normalisation drops leading zeros: 2026.08.27 -> 2026.8.27.
normalised = ".".join(
str(int(part)) if part.isdigit() else part
for part in declared.split(".")
)
assert informix_db.__version__ in {declared, normalised}
def test_init_looks_up_the_declared_distribution_name(pyproject: dict) -> None:
"""Catch the rename trap directly: the name passed to
``importlib.metadata.version()`` must equal ``[project].name``."""
source = (
Path(informix_db.__file__).resolve().parent / "__init__.py"
).read_text(encoding="utf-8")
declared_name = pyproject["project"]["name"]
assert f'version("{declared_name}")' in source, (
f"informix_db/__init__.py should call version({declared_name!r}) to "
"match [project].name in pyproject.toml"
)
def test_version_is_importable_and_stringy() -> None:
assert isinstance(informix_db.__version__, str)
assert informix_db.__version__
def test_public_surface_is_exported() -> None:
for name in informix_db.__all__:
assert hasattr(informix_db, name), f"{name} in __all__ but not defined"