Ryan Malloy 6afdbcabb3 Phase 27: Wire lock + async cancellation eviction (2026.05.05.1)
Closes Hamilton audit Critical #2 (concurrency / wire lock) and
High #3 (async cancellation evicts cleanly). Phase 26 fixed what
gets returned to the pool; Phase 27 fixes what can interleave on
the wire while it's running.

What changed:

connections.py:
* Added Connection._wire_lock = threading.RLock(). Wrapped commit(),
  rollback(), fast_path_call() under the lock.
* _ensure_transaction documents the lock as a precondition AND
  asserts ownership at runtime (_wire_lock._is_owned()) so a future
  caller adding a third call site fails loudly.
* close() tries to acquire wire lock with 0.5s timeout before
  SQ_EXIT; skips polite exit and force-closes if busy.

cursors.py:
* execute() body extracted into _execute_under_wire_lock() and
  called under the lock.
* executemany() body wrapped inline.
* _sfetch_at() wrapped - covers all scrollable fetch_* methods
  that delegate to it.
* close() locks the CLOSE+RELEASE for scrollable cursors.

pool.py:
* release() acquires conn._wire_lock with 5s timeout before rollback.
  On timeout: log WARNING, evict connection. Constant
  _RELEASE_WIRE_LOCK_TIMEOUT for tunability.

aio.py:
* AsyncConnectionPool.connection() now catches CancelledError /
  TimeoutError separately and routes to broken=True. Combined with
  the wire lock, asyncio.wait_for around aio DB calls is now safe.
* Updated docstring; mirrored in docs/USAGE.md.

Margaret Hamilton review surfaced three actionable conditions, all
addressed before tagging:
* Cancellation test used contextlib.suppress - could pass without
  exercising the cancellation path on a fast runner. Switched to
  pytest.raises so the test fails if timeout doesn't fire.
* _ensure_transaction precondition documented but unchecked at
  runtime. Added assert self._wire_lock._is_owned() guard.
* Connection.close() was unsynchronized. Now tries 0.5s acquire
  before SQ_EXIT.

Two new regression tests in tests/test_pool.py:
* test_concurrent_threads_on_one_connection_dont_interleave_pdus
  (without lock: garbled results / hangs)
* test_async_wait_for_cancellation_evicts_connection
  (asserts pool size shrinks; cancellation actually fires)

72 unit + 228 integration + 28 benchmark = 328 tests; ruff clean.

Hamilton verdict: PRODUCTION READY WITH CAVEATS (was) -> CAVEATS
NARROWED FURTHER (now). 0 critical, 2 high remaining (cursor
finalizers + bare-except in error drain) - both Phase 28 scope.
2026-05-05 03:40:39 -06:00

informix-db

Pure-Python driver for IBM Informix IDS, speaking the SQLI wire protocol over raw sockets. No IBM Client SDK. No JVM. No native libraries. PEP 249 compliant; sync + async APIs; built-in connection pool; TLS support.

To our knowledge this is the first pure-socket Informix driver in any language — every other Informix driver (IfxPy, the legacy informixdb, ODBC bridges, JPype/JDBC, Perl DBD::Informix) wraps either IBM's CSDK or the JDBC JAR.

pip install informix-db

Quick start

import informix_db

with informix_db.connect(
    host="db.example.com", port=9088,
    user="informix", password="...",
    database="mydb", server="informix",
) as conn:
    cur = conn.cursor()
    cur.execute("SELECT id, name FROM users WHERE id = ?", (42,))
    user_id, name = cur.fetchone()

Async (FastAPI / aiohttp / asyncio)

import asyncio
from informix_db import aio

async def main():
    pool = await aio.create_pool(
        host="db.example.com", user="informix", password="...",
        database="mydb",
        min_size=1, max_size=10,
    )
    async with pool.connection() as conn:
        cur = await conn.cursor()
        await cur.execute("SELECT id, name FROM users WHERE id = ?", (42,))
        row = await cur.fetchone()
    await pool.close()

asyncio.run(main())

Connection pool (sync)

import informix_db

pool = informix_db.create_pool(
    host="db.example.com", user="informix", password="...",
    database="mydb",
    min_size=1, max_size=10, acquire_timeout=5.0,
)

with pool.connection() as conn:
    cur = conn.cursor()
    cur.execute("...")

pool.close()

TLS

import ssl

# Production: bring your own context
ctx = ssl.create_default_context(cafile="/path/to/ca.pem")
informix_db.connect(host="...", port=9089, ..., tls=ctx)

# Dev / self-signed: tls=True disables verification
informix_db.connect(host="127.0.0.1", port=9089, ..., tls=True)

Informix uses dedicated TLS-enabled listener ports (configured server-side in sqlhosts) rather than STARTTLS upgrade — point port at the TLS listener (often 9089) when tls is enabled.

Type support

SQL type Python type
SMALLINT / INT / BIGINT / SERIAL int
FLOAT / SMALLFLOAT float
DECIMAL(p,s) / MONEY decimal.Decimal
CHAR / VARCHAR / NCHAR / NVCHAR / LVARCHAR str
BOOLEAN bool
DATE datetime.date
DATETIME YEAR TO ... datetime.datetime / datetime.time / datetime.date
INTERVAL DAY TO FRACTION datetime.timedelta
INTERVAL YEAR TO MONTH informix_db.IntervalYM
BYTE / TEXT (legacy in-row blobs) bytes / str
BLOB / CLOB (smart-LOBs) informix_db.BlobLocator / informix_db.ClobLocator (read via cursor.read_blob_column, write via cursor.write_blob_column)
ROW(...) informix_db.RowValue
SET(...) / MULTISET(...) / LIST(...) informix_db.CollectionValue
NULL None

Smart-LOB (BLOB / CLOB) read & write

# Read: returns the actual bytes
data = cur.read_blob_column(
    "SELECT data FROM photos WHERE id = ?", (42,)
)

# Write: BLOB_PLACEHOLDER token marks where the BLOB goes
cur.write_blob_column(
    "INSERT INTO photos VALUES (?, BLOB_PLACEHOLDER)",
    blob_data=jpeg_bytes,
    params=(42,),
)

Both work end-to-end in pure Python via the lotofile / filetoblob server functions intercepted at the SQ_FILE (98) wire-protocol level — no thread of native machinery. See docs/DECISION_LOG.md §1011 for the architecture pivot that made this possible.

Direct stored-procedure invocation (fast-path)

# Cleanly close a smart-LOB descriptor opened via SQL
result = conn.fast_path_call(
    "function informix.ifx_lo_close(integer)", lofd
)
# result == [0] on success

The fast-path RPC (SQ_FPROUTINE / SQ_EXFPROUTINE) bypasses PREPARE → EXECUTE → FETCH for direct UDF/SPL calls. Routine handles are cached per-connection, so repeated calls to the same function take a single round-trip.

Server compatibility

Tested against IBM Informix Dynamic Server 15.0.1.0.3DE (the official icr.io/informix/informix-developer-database Docker image). The wire protocol is stable across modern Informix versions; should work against 12.10+ unmodified.

For features that need server-side configuration (smart-LOBs, logged transactions), see docs/DECISION_LOG.md:

  • Phase 7 — logged-DB transactions
  • Phase 8 — BYTE/TEXT (needs blobspace)
  • Phase 10/11 — BLOB/CLOB (needs sbspace + SBSPACENAME config + level-0 archive)

Standards & guarantees

  • PEP 249 (DB-API 2.0): connect(), Connection, Cursor, description, rowcount, exception hierarchy
  • paramstyle = "numeric" (Informix's native ESQL/C convention; ? and :1 both work)
  • Threadsafety = 1: threads may share the module but not connections; the pool gives per-thread connection access
  • CalVer versioning: YYYY.MM.DD releases. PEP 440 post-releases (.1, .2) for same-day fixes.

Development

# Set up the dev environment
uv sync --dev

# Run the test suite (unit-only by default; no Docker needed)
uv run pytest                       # 69 unit tests
uv run pytest -m integration        # 163 integration tests (needs Docker)

# Lint
uv run ruff check src/ tests/

The integration suite expects an Informix Developer Edition container on localhost:9088:

docker compose -f tests/docker-compose.yml up -d

For the smart-LOB tests specifically, the dev container needs additional one-time setup (blobspace + sbspace + level-0 archive). See docs/DECISION_LOG.md §10 for the exact onspaces / onmode / ontape commands.

Documentation

  • docs/USAGE.md — practical recipes: connections, parameter binding, type mapping, transactions, performance tips, scrollable cursors, BLOBs, async, TLS, locale/Unicode, error handling, known limitations
  • tests/benchmarks/README.md — performance baselines, headline numbers, how to run regressions
  • CHANGELOG.md — phase-by-phase release notes

Project history & design rationale

This driver was built incrementally across 22+ phases, each with a focused scope and decision log. The reasoning trail lives in:

Notable architectural pivots documented in the decision log:

  • Phase 10/11 (smart-LOB read/write): used lotofile/filetoblob SQL functions + SQ_FILE protocol intercept instead of the heavier SQ_FPROUTINE + SQ_LODATA stack — ~3x smaller than originally projected
  • Phase 7 (logged-DB transactions): discovered Informix requires explicit SQ_BEGIN before each transaction in non-ANSI mode, plus SQ_RBWORK needs a savepoint short payload
  • Phase 16 (async): shipped thread-pool wrapping (~250 lines) instead of full I/O abstraction refactor (~2000 lines); functionally equivalent for typical FastAPI workloads

License

MIT.

Description
Pure-Python driver for IBM Informix IDS — speaks the SQLI wire protocol over a raw socket. No CSDK, no JVM, no native libraries.
https://informix-db.warehack.ing
Readme MIT 716 KiB
Languages
Python 85.6%
MDX 8.1%
CSS 2.1%
Java 1.7%
Astro 1%
Other 1.4%