Ryan Malloy fdb9ba32d5 Phase 28: Resource leak hardening (2026.05.05.2)
Closes Hamilton audit High #4 (bare-except in error drain) and
High #5 (no cursor finalizers), plus 1 medium one-liner.

After Phases 26-28, 0 CRITICAL and 0 HIGH audit findings remain.
Driver is PRODUCTION READY.

What changed:

cursors.py:
* Cursor finalizers via weakref.finalize. Mid-fetch raises (or any
  GC without explicit close()) now release server-side resources
  (CLOSE + RELEASE PDUs). Pre-built static PDU bytes at module load
  so finalizer can run on any thread without allocating or calling
  cursor methods.
* Non-blocking lock acquire prevents cross-thread GC deadlock.
  WARNING log on lock-busy so leak accumulation is visible.
* state=[False] list pattern keeps finalizer closure weak. GIL
  dependency of atomic single-element mutation documented.
* _raise_sq_err near-token parse: (ProtocolError, OSError) only.
* _raise_sq_err drain: force-close connection on same exceptions
  (wire unrecoverable after desync).

connections.py:
* _raise_sq_err drain: same hardening as cursor version. Force-close
  on (ProtocolError, OSError, OperationalError) - the latter from
  _drain_to_eot raising on unknown tags. Documented inline.
* Added contextlib import for force-close suppression.

cursors.py write_blob_column:
* BLOB_PLACEHOLDER validation now requires EXACTLY ONE occurrence.
  Pre-Phase-28, str.replace silently substituted every occurrence -
  corrupting SQL containing the literal string in comments etc.
  Now raises ProgrammingError with workaround pointer.

_resultset.py:
* Investigated end-of-loop bounds check for parse_tuple_payload.
  Reverted: long-standing off-by-one in UDTVAR(lvarchar) trailing-
  pad logic produces benign over-reads (payload is a fully-extracted
  bytes object; over-reads return empty slices through unused
  branches). Real silent-corruption surfaces are length-prefix
  decoders, needing branch-local checks. Documented as deliberate
  non-fix.

Margaret Hamilton review surfaced two blocking conditions:

* Asymmetric failure handling: _raise_sq_err force-closed the
  connection on wire desync, but the cursor finalizer silently
  swallowed identical failures. "Same wire, same failure mode,
  same response" - finalizer now matches _raise_sq_err's discipline.

* Leak visibility: wire-lock-busy log was DEBUG. Promoted to WARNING
  so leak accumulation on pooled connections is visible.

Plus three documentation improvements (GIL dependency, OperationalError
in desync taxonomy, parse_tuple non-fix rationale).

One new regression test:
* test_write_blob_column_rejects_multiple_placeholders

72 unit + 229 integration + 28 benchmark = 329 tests; ruff clean.

Phase 29 ticket (Hamilton recommended): deferred-cleanup queue
drained at next _send_pdu, closes unbounded-leak gap on long-lived
pooled connections. Not blocking Phase 28.

Hamilton audit verdict:
  Pre-26:  2 critical, 3 high, 5 medium
  Post-28: 0 critical, 0 high, 4 medium
2026-05-05 03:56:24 -06:00

informix-db

Pure-Python driver for IBM Informix IDS, speaking the SQLI wire protocol over raw sockets. No IBM Client SDK. No JVM. No native libraries. PEP 249 compliant; sync + async APIs; built-in connection pool; TLS support.

To our knowledge this is the first pure-socket Informix driver in any language — every other Informix driver (IfxPy, the legacy informixdb, ODBC bridges, JPype/JDBC, Perl DBD::Informix) wraps either IBM's CSDK or the JDBC JAR.

pip install informix-db

Quick start

import informix_db

with informix_db.connect(
    host="db.example.com", port=9088,
    user="informix", password="...",
    database="mydb", server="informix",
) as conn:
    cur = conn.cursor()
    cur.execute("SELECT id, name FROM users WHERE id = ?", (42,))
    user_id, name = cur.fetchone()

Async (FastAPI / aiohttp / asyncio)

import asyncio
from informix_db import aio

async def main():
    pool = await aio.create_pool(
        host="db.example.com", user="informix", password="...",
        database="mydb",
        min_size=1, max_size=10,
    )
    async with pool.connection() as conn:
        cur = await conn.cursor()
        await cur.execute("SELECT id, name FROM users WHERE id = ?", (42,))
        row = await cur.fetchone()
    await pool.close()

asyncio.run(main())

Connection pool (sync)

import informix_db

pool = informix_db.create_pool(
    host="db.example.com", user="informix", password="...",
    database="mydb",
    min_size=1, max_size=10, acquire_timeout=5.0,
)

with pool.connection() as conn:
    cur = conn.cursor()
    cur.execute("...")

pool.close()

TLS

import ssl

# Production: bring your own context
ctx = ssl.create_default_context(cafile="/path/to/ca.pem")
informix_db.connect(host="...", port=9089, ..., tls=ctx)

# Dev / self-signed: tls=True disables verification
informix_db.connect(host="127.0.0.1", port=9089, ..., tls=True)

Informix uses dedicated TLS-enabled listener ports (configured server-side in sqlhosts) rather than STARTTLS upgrade — point port at the TLS listener (often 9089) when tls is enabled.

Type support

SQL type Python type
SMALLINT / INT / BIGINT / SERIAL int
FLOAT / SMALLFLOAT float
DECIMAL(p,s) / MONEY decimal.Decimal
CHAR / VARCHAR / NCHAR / NVCHAR / LVARCHAR str
BOOLEAN bool
DATE datetime.date
DATETIME YEAR TO ... datetime.datetime / datetime.time / datetime.date
INTERVAL DAY TO FRACTION datetime.timedelta
INTERVAL YEAR TO MONTH informix_db.IntervalYM
BYTE / TEXT (legacy in-row blobs) bytes / str
BLOB / CLOB (smart-LOBs) informix_db.BlobLocator / informix_db.ClobLocator (read via cursor.read_blob_column, write via cursor.write_blob_column)
ROW(...) informix_db.RowValue
SET(...) / MULTISET(...) / LIST(...) informix_db.CollectionValue
NULL None

Smart-LOB (BLOB / CLOB) read & write

# Read: returns the actual bytes
data = cur.read_blob_column(
    "SELECT data FROM photos WHERE id = ?", (42,)
)

# Write: BLOB_PLACEHOLDER token marks where the BLOB goes
cur.write_blob_column(
    "INSERT INTO photos VALUES (?, BLOB_PLACEHOLDER)",
    blob_data=jpeg_bytes,
    params=(42,),
)

Both work end-to-end in pure Python via the lotofile / filetoblob server functions intercepted at the SQ_FILE (98) wire-protocol level — no thread of native machinery. See docs/DECISION_LOG.md §1011 for the architecture pivot that made this possible.

Direct stored-procedure invocation (fast-path)

# Cleanly close a smart-LOB descriptor opened via SQL
result = conn.fast_path_call(
    "function informix.ifx_lo_close(integer)", lofd
)
# result == [0] on success

The fast-path RPC (SQ_FPROUTINE / SQ_EXFPROUTINE) bypasses PREPARE → EXECUTE → FETCH for direct UDF/SPL calls. Routine handles are cached per-connection, so repeated calls to the same function take a single round-trip.

Server compatibility

Tested against IBM Informix Dynamic Server 15.0.1.0.3DE (the official icr.io/informix/informix-developer-database Docker image). The wire protocol is stable across modern Informix versions; should work against 12.10+ unmodified.

For features that need server-side configuration (smart-LOBs, logged transactions), see docs/DECISION_LOG.md:

  • Phase 7 — logged-DB transactions
  • Phase 8 — BYTE/TEXT (needs blobspace)
  • Phase 10/11 — BLOB/CLOB (needs sbspace + SBSPACENAME config + level-0 archive)

Standards & guarantees

  • PEP 249 (DB-API 2.0): connect(), Connection, Cursor, description, rowcount, exception hierarchy
  • paramstyle = "numeric" (Informix's native ESQL/C convention; ? and :1 both work)
  • Threadsafety = 1: threads may share the module but not connections; the pool gives per-thread connection access
  • CalVer versioning: YYYY.MM.DD releases. PEP 440 post-releases (.1, .2) for same-day fixes.

Development

# Set up the dev environment
uv sync --dev

# Run the test suite (unit-only by default; no Docker needed)
uv run pytest                       # 69 unit tests
uv run pytest -m integration        # 163 integration tests (needs Docker)

# Lint
uv run ruff check src/ tests/

The integration suite expects an Informix Developer Edition container on localhost:9088:

docker compose -f tests/docker-compose.yml up -d

For the smart-LOB tests specifically, the dev container needs additional one-time setup (blobspace + sbspace + level-0 archive). See docs/DECISION_LOG.md §10 for the exact onspaces / onmode / ontape commands.

Documentation

  • docs/USAGE.md — practical recipes: connections, parameter binding, type mapping, transactions, performance tips, scrollable cursors, BLOBs, async, TLS, locale/Unicode, error handling, known limitations
  • tests/benchmarks/README.md — performance baselines, headline numbers, how to run regressions
  • CHANGELOG.md — phase-by-phase release notes

Project history & design rationale

This driver was built incrementally across 22+ phases, each with a focused scope and decision log. The reasoning trail lives in:

Notable architectural pivots documented in the decision log:

  • Phase 10/11 (smart-LOB read/write): used lotofile/filetoblob SQL functions + SQ_FILE protocol intercept instead of the heavier SQ_FPROUTINE + SQ_LODATA stack — ~3x smaller than originally projected
  • Phase 7 (logged-DB transactions): discovered Informix requires explicit SQ_BEGIN before each transaction in non-ANSI mode, plus SQ_RBWORK needs a savepoint short payload
  • Phase 16 (async): shipped thread-pool wrapping (~250 lines) instead of full I/O abstraction refactor (~2000 lines); functionally equivalent for typical FastAPI workloads

License

MIT.

Description
Pure-Python driver for IBM Informix IDS — speaks the SQLI wire protocol over a raw socket. No CSDK, no JVM, no native libraries.
https://informix-db.warehack.ing
Readme MIT 716 KiB
Languages
Python 85.6%
MDX 8.1%
CSS 2.1%
Java 1.7%
Astro 1%
Other 1.4%