Ryan Malloy
0e0dfcba26
The docs/USAGE.md predated Phases 17-21, so anyone landing on PyPI was
missing scrollable cursors, locale/Unicode, the autocommit cliff
finding, and the type-mapping reference.
Added sections to docs/USAGE.md:
* Locale and Unicode - client_locale, Connection.encoding, CLIENT_LOCALE
vs DB_LOCALE, when characters can't fit the codec
* Type mapping reference - full SQL <-> Python type table, NULL
sentinels subsection, IntervalYM
* Performance tips - 53x autocommit-cliff fix, 100x executemany win,
72x pool win, with the actual benchmark numbers from Phase 21.1
* Scrollable cursors - fetch_* API, in-memory vs server-side trade-off,
edge cases (past-end semantics, negative indexing, rownumber)
* Timeouts and keepalive subsection - production starting points
* Environment dictionary subsection - env={} parameter
* Known limitations - explicit table of what doesn't work (named
params, complex UDT bind, GSSAPI, XA) with workarounds; "things
that might surprise you" notes
README.md - added Documentation section linking to docs/USAGE.md
and tests/benchmarks/README.md.
Doc corrections caught during review:
* cursor.rownumber is 0-indexed (impl has always been correct; only
the original docstring wording was loose)
* fetch_* methods work on BOTH scrollable=True and default cursors;
the in-memory path supports them too
USAGE.md grew from 345 lines to 633.
190 lines
7.3 KiB
Markdown
190 lines
7.3 KiB
Markdown
# informix-db
|
||
|
||
Pure-Python driver for IBM Informix IDS, speaking the SQLI wire protocol over raw sockets. **No IBM Client SDK. No JVM. No native libraries.** PEP 249 compliant; sync + async APIs; built-in connection pool; TLS support.
|
||
|
||
To our knowledge this is the **first pure-socket Informix driver in any language** — every other Informix driver (`IfxPy`, the legacy `informixdb`, ODBC bridges, JPype/JDBC, Perl `DBD::Informix`) wraps either IBM's CSDK or the JDBC JAR.
|
||
|
||
```bash
|
||
pip install informix-db
|
||
```
|
||
|
||
## Quick start
|
||
|
||
```python
|
||
import informix_db
|
||
|
||
with informix_db.connect(
|
||
host="db.example.com", port=9088,
|
||
user="informix", password="...",
|
||
database="mydb", server="informix",
|
||
) as conn:
|
||
cur = conn.cursor()
|
||
cur.execute("SELECT id, name FROM users WHERE id = ?", (42,))
|
||
user_id, name = cur.fetchone()
|
||
```
|
||
|
||
## Async (FastAPI / aiohttp / asyncio)
|
||
|
||
```python
|
||
import asyncio
|
||
from informix_db import aio
|
||
|
||
async def main():
|
||
pool = await aio.create_pool(
|
||
host="db.example.com", user="informix", password="...",
|
||
database="mydb",
|
||
min_size=1, max_size=10,
|
||
)
|
||
async with pool.connection() as conn:
|
||
cur = await conn.cursor()
|
||
await cur.execute("SELECT id, name FROM users WHERE id = ?", (42,))
|
||
row = await cur.fetchone()
|
||
await pool.close()
|
||
|
||
asyncio.run(main())
|
||
```
|
||
|
||
## Connection pool (sync)
|
||
|
||
```python
|
||
import informix_db
|
||
|
||
pool = informix_db.create_pool(
|
||
host="db.example.com", user="informix", password="...",
|
||
database="mydb",
|
||
min_size=1, max_size=10, acquire_timeout=5.0,
|
||
)
|
||
|
||
with pool.connection() as conn:
|
||
cur = conn.cursor()
|
||
cur.execute("...")
|
||
|
||
pool.close()
|
||
```
|
||
|
||
## TLS
|
||
|
||
```python
|
||
import ssl
|
||
|
||
# Production: bring your own context
|
||
ctx = ssl.create_default_context(cafile="/path/to/ca.pem")
|
||
informix_db.connect(host="...", port=9089, ..., tls=ctx)
|
||
|
||
# Dev / self-signed: tls=True disables verification
|
||
informix_db.connect(host="127.0.0.1", port=9089, ..., tls=True)
|
||
```
|
||
|
||
Informix uses dedicated TLS-enabled listener ports (configured server-side in `sqlhosts`) rather than STARTTLS upgrade — point `port` at the TLS listener (often `9089`) when `tls` is enabled.
|
||
|
||
## Type support
|
||
|
||
| SQL type | Python type |
|
||
|---|---|
|
||
| `SMALLINT` / `INT` / `BIGINT` / `SERIAL` | `int` |
|
||
| `FLOAT` / `SMALLFLOAT` | `float` |
|
||
| `DECIMAL(p,s)` / `MONEY` | `decimal.Decimal` |
|
||
| `CHAR` / `VARCHAR` / `NCHAR` / `NVCHAR` / `LVARCHAR` | `str` |
|
||
| `BOOLEAN` | `bool` |
|
||
| `DATE` | `datetime.date` |
|
||
| `DATETIME YEAR TO ...` | `datetime.datetime` / `datetime.time` / `datetime.date` |
|
||
| `INTERVAL DAY TO FRACTION` | `datetime.timedelta` |
|
||
| `INTERVAL YEAR TO MONTH` | `informix_db.IntervalYM` |
|
||
| `BYTE` / `TEXT` (legacy in-row blobs) | `bytes` / `str` |
|
||
| `BLOB` / `CLOB` (smart-LOBs) | `informix_db.BlobLocator` / `informix_db.ClobLocator` (read via `cursor.read_blob_column`, write via `cursor.write_blob_column`) |
|
||
| `ROW(...)` | `informix_db.RowValue` |
|
||
| `SET(...)` / `MULTISET(...)` / `LIST(...)` | `informix_db.CollectionValue` |
|
||
| `NULL` | `None` |
|
||
|
||
## Smart-LOB (BLOB / CLOB) read & write
|
||
|
||
```python
|
||
# Read: returns the actual bytes
|
||
data = cur.read_blob_column(
|
||
"SELECT data FROM photos WHERE id = ?", (42,)
|
||
)
|
||
|
||
# Write: BLOB_PLACEHOLDER token marks where the BLOB goes
|
||
cur.write_blob_column(
|
||
"INSERT INTO photos VALUES (?, BLOB_PLACEHOLDER)",
|
||
blob_data=jpeg_bytes,
|
||
params=(42,),
|
||
)
|
||
```
|
||
|
||
Both work end-to-end in pure Python via the `lotofile` / `filetoblob` server functions intercepted at the `SQ_FILE` (98) wire-protocol level — no thread of native machinery. See [`docs/DECISION_LOG.md`](docs/DECISION_LOG.md) §10–11 for the architecture pivot that made this possible.
|
||
|
||
## Direct stored-procedure invocation (fast-path)
|
||
|
||
```python
|
||
# Cleanly close a smart-LOB descriptor opened via SQL
|
||
result = conn.fast_path_call(
|
||
"function informix.ifx_lo_close(integer)", lofd
|
||
)
|
||
# result == [0] on success
|
||
```
|
||
|
||
The fast-path RPC (`SQ_FPROUTINE` / `SQ_EXFPROUTINE`) bypasses PREPARE → EXECUTE → FETCH for direct UDF/SPL calls. Routine handles are cached per-connection, so repeated calls to the same function take a single round-trip.
|
||
|
||
## Server compatibility
|
||
|
||
Tested against IBM Informix Dynamic Server **15.0.1.0.3DE** (the official `icr.io/informix/informix-developer-database` Docker image). The wire protocol is stable across modern Informix versions; should work against 12.10+ unmodified.
|
||
|
||
For features that need server-side configuration (smart-LOBs, logged transactions), see [`docs/DECISION_LOG.md`](docs/DECISION_LOG.md):
|
||
- Phase 7 — logged-DB transactions
|
||
- Phase 8 — BYTE/TEXT (needs blobspace)
|
||
- Phase 10/11 — BLOB/CLOB (needs sbspace + `SBSPACENAME` config + level-0 archive)
|
||
|
||
## Standards & guarantees
|
||
|
||
* **PEP 249** (DB-API 2.0): `connect()`, `Connection`, `Cursor`, `description`, `rowcount`, exception hierarchy
|
||
* **`paramstyle = "numeric"`** (Informix's native ESQL/C convention; `?` and `:1` both work)
|
||
* **Threadsafety = 1**: threads may share the module but not connections; the pool gives per-thread connection access
|
||
* **CalVer versioning**: `YYYY.MM.DD` releases. PEP 440 post-releases (`.1`, `.2`) for same-day fixes.
|
||
|
||
## Development
|
||
|
||
```bash
|
||
# Set up the dev environment
|
||
uv sync --dev
|
||
|
||
# Run the test suite (unit-only by default; no Docker needed)
|
||
uv run pytest # 69 unit tests
|
||
uv run pytest -m integration # 163 integration tests (needs Docker)
|
||
|
||
# Lint
|
||
uv run ruff check src/ tests/
|
||
```
|
||
|
||
The integration suite expects an Informix Developer Edition container on `localhost:9088`:
|
||
|
||
```bash
|
||
docker compose -f tests/docker-compose.yml up -d
|
||
```
|
||
|
||
For the smart-LOB tests specifically, the dev container needs additional one-time setup (blobspace + sbspace + level-0 archive). See [`docs/DECISION_LOG.md`](docs/DECISION_LOG.md) §10 for the exact `onspaces` / `onmode` / `ontape` commands.
|
||
|
||
## Documentation
|
||
|
||
- [**`docs/USAGE.md`**](docs/USAGE.md) — practical recipes: connections, parameter binding, type mapping, transactions, performance tips, scrollable cursors, BLOBs, async, TLS, locale/Unicode, error handling, known limitations
|
||
- [`tests/benchmarks/README.md`](tests/benchmarks/README.md) — performance baselines, headline numbers, how to run regressions
|
||
- `CHANGELOG.md` — phase-by-phase release notes
|
||
|
||
## Project history & design rationale
|
||
|
||
This driver was built incrementally across 22+ phases, each with a focused scope and decision log. The reasoning trail lives in:
|
||
|
||
- [`docs/PROTOCOL_NOTES.md`](docs/PROTOCOL_NOTES.md) — byte-level SQLI wire-format reference
|
||
- [`docs/JDBC_NOTES.md`](docs/JDBC_NOTES.md) — index into the decompiled IBM JDBC driver, used as a clean-room reference
|
||
- [`docs/DECISION_LOG.md`](docs/DECISION_LOG.md) — phase-by-phase architectural decisions, with the *why* preserved
|
||
- [`docs/CAPTURES/`](docs/CAPTURES/) — annotated socat hex-dump captures
|
||
|
||
Notable architectural pivots documented in the decision log:
|
||
- **Phase 10/11** (smart-LOB read/write): used `lotofile`/`filetoblob` SQL functions + `SQ_FILE` protocol intercept instead of the heavier `SQ_FPROUTINE` + `SQ_LODATA` stack — ~3x smaller than originally projected
|
||
- **Phase 7** (logged-DB transactions): discovered Informix requires explicit `SQ_BEGIN` before each transaction in non-ANSI mode, plus `SQ_RBWORK` needs a savepoint short payload
|
||
- **Phase 16** (async): shipped thread-pool wrapping (~250 lines) instead of full I/O abstraction refactor (~2000 lines); functionally equivalent for typical FastAPI workloads
|
||
|
||
## License
|
||
|
||
MIT.
|