Closes Hamilton audit Critical #2 (concurrency / wire lock) and High #3 (async cancellation evicts cleanly). Phase 26 fixed what gets returned to the pool; Phase 27 fixes what can interleave on the wire while it's running. What changed: connections.py: * Added Connection._wire_lock = threading.RLock(). Wrapped commit(), rollback(), fast_path_call() under the lock. * _ensure_transaction documents the lock as a precondition AND asserts ownership at runtime (_wire_lock._is_owned()) so a future caller adding a third call site fails loudly. * close() tries to acquire wire lock with 0.5s timeout before SQ_EXIT; skips polite exit and force-closes if busy. cursors.py: * execute() body extracted into _execute_under_wire_lock() and called under the lock. * executemany() body wrapped inline. * _sfetch_at() wrapped - covers all scrollable fetch_* methods that delegate to it. * close() locks the CLOSE+RELEASE for scrollable cursors. pool.py: * release() acquires conn._wire_lock with 5s timeout before rollback. On timeout: log WARNING, evict connection. Constant _RELEASE_WIRE_LOCK_TIMEOUT for tunability. aio.py: * AsyncConnectionPool.connection() now catches CancelledError / TimeoutError separately and routes to broken=True. Combined with the wire lock, asyncio.wait_for around aio DB calls is now safe. * Updated docstring; mirrored in docs/USAGE.md. Margaret Hamilton review surfaced three actionable conditions, all addressed before tagging: * Cancellation test used contextlib.suppress - could pass without exercising the cancellation path on a fast runner. Switched to pytest.raises so the test fails if timeout doesn't fire. * _ensure_transaction precondition documented but unchecked at runtime. Added assert self._wire_lock._is_owned() guard. * Connection.close() was unsynchronized. Now tries 0.5s acquire before SQ_EXIT. Two new regression tests in tests/test_pool.py: * test_concurrent_threads_on_one_connection_dont_interleave_pdus (without lock: garbled results / hangs) * test_async_wait_for_cancellation_evicts_connection (asserts pool size shrinks; cancellation actually fires) 72 unit + 228 integration + 28 benchmark = 328 tests; ruff clean. Hamilton verdict: PRODUCTION READY WITH CAVEATS (was) -> CAVEATS NARROWED FURTHER (now). 0 critical, 2 high remaining (cursor finalizers + bare-except in error drain) - both Phase 28 scope.
651 lines
26 KiB
Markdown
651 lines
26 KiB
Markdown
# Usage Guide
|
||
|
||
Practical recipes for common Informix patterns with `informix-db`. For installation and a quick overview, see the [README](../README.md). For protocol-level / architectural decisions, see the [DECISION_LOG](DECISION_LOG.md).
|
||
|
||
## Connecting
|
||
|
||
```python
|
||
import informix_db
|
||
|
||
conn = informix_db.connect(
|
||
host="db.example.com",
|
||
port=9088,
|
||
user="informix",
|
||
password="...",
|
||
database="mydb",
|
||
server="informix", # the DBSERVERNAME from sqlhosts
|
||
autocommit=False, # default; opt-in with True
|
||
connect_timeout=10.0, # seconds; None = OS default
|
||
read_timeout=30.0, # seconds for each read; None = no timeout
|
||
keepalive=False, # SO_KEEPALIVE on the socket
|
||
client_locale="en_US.8859-1",
|
||
)
|
||
```
|
||
|
||
`server` is **not** the hostname — it's the Informix DBSERVERNAME the listener identifies itself as (configured server-side in `$ONCONFIG`'s `DBSERVERNAME`). For the official IBM Developer Edition Docker image, the default `"informix"` is correct.
|
||
|
||
`database` may be `None` to log in without selecting a database; the server still completes a successful login. Useful for cross-database queries that fully qualify table names.
|
||
|
||
### Timeouts and keepalive
|
||
|
||
| Parameter | Purpose | Default |
|
||
|---|---|---|
|
||
| `connect_timeout` | Time-bound for the TCP connect + login handshake. `None` uses the OS default (typically minutes). | `None` |
|
||
| `read_timeout` | Per-read timeout on subsequent socket reads. Fires `OperationalError` on a hung server. | `None` |
|
||
| `keepalive` | Set `SO_KEEPALIVE` on the socket. Useful for long-lived idle connections behind aggressive NAT/firewalls. | `False` |
|
||
|
||
A reasonable production starting point: `connect_timeout=10.0, read_timeout=30.0, keepalive=True`. The connect timeout protects startup; the read timeout protects against a frozen server; keepalive protects against silent idle disconnection.
|
||
|
||
### Environment dictionary
|
||
|
||
The `env={}` parameter sets server-side session variables sent in the login PDU. Useful for things like `OPTOFC` (optimize-on-fetch-close), `IFX_AUTOFREE`, `OPT_GOAL`, or any other runtime knob the server reads from the session env block.
|
||
|
||
```python
|
||
informix_db.connect(
|
||
...,
|
||
env={
|
||
"OPT_GOAL": "-1", # optimize for first-row return
|
||
"OPTOFC": "1", # auto-free cursors at fetch-close
|
||
"IFX_AUTOFREE": "1",
|
||
},
|
||
)
|
||
```
|
||
|
||
`CLIENT_LOCALE` is set automatically from the `client_locale=` parameter — don't put it in `env=`.
|
||
|
||
## Locale and Unicode
|
||
|
||
The connection's `client_locale` controls how Python `str` values are encoded to bytes (and back) for CHAR / VARCHAR / NCHAR / NVCHAR / LVARCHAR / CLOB columns. The default `"en_US.8859-1"` is safe for ASCII + Western European text. **For multibyte text (CJK, Cyrillic, Arabic, emoji), set `client_locale="en_US.utf8"` AND make sure the database's `DB_LOCALE` is also UTF-8.**
|
||
|
||
```python
|
||
conn = informix_db.connect(..., client_locale="en_US.utf8")
|
||
print(conn.encoding) # "utf-8"
|
||
|
||
cur = conn.cursor()
|
||
cur.execute("INSERT INTO docs (body) VALUES (?)", ("你好世界",))
|
||
```
|
||
|
||
The `Connection.encoding` property reports the resolved Python codec name. Common mappings:
|
||
|
||
| Locale | Python codec |
|
||
|---|---|
|
||
| `en_US.8859-1` (default) | `iso-8859-1` |
|
||
| `en_US.utf8` / `en_US.UTF-8` | `utf-8` |
|
||
| `en_US.8859-15` | `iso-8859-15` |
|
||
| Anything without a codeset suffix, or unknown | falls back to `iso-8859-1` |
|
||
|
||
### CLIENT_LOCALE vs DB_LOCALE
|
||
|
||
* **`CLIENT_LOCALE`** is what *your code* uses to encode / decode string parameters and column values. Set per-connection.
|
||
* **`DB_LOCALE`** is what *the database* uses to store string columns. Set at `CREATE DATABASE` time, immutable afterwards.
|
||
|
||
If they match, no transcoding happens. If they differ, the server transcodes between them at the storage boundary — and any character in your data that doesn't exist in `DB_LOCALE`'s codeset is either replaced with `?` (lossy) or rejected with sqlcode `-1820` (depends on the server version). The IBM Developer Edition Docker image's default `testdb` is created with `DB_LOCALE=en_US.8859-1`; storing `"你好"` there will fail server-side regardless of `CLIENT_LOCALE`.
|
||
|
||
To create a UTF-8 database for full multibyte support:
|
||
|
||
```bash
|
||
# Inside the container, before CREATE DATABASE:
|
||
export DB_LOCALE=en_US.utf8
|
||
export CLIENT_LOCALE=en_US.utf8
|
||
```
|
||
|
||
```sql
|
||
CREATE DATABASE my_utf8db WITH LOG IN rootdbs;
|
||
```
|
||
|
||
### When characters can't fit the codec
|
||
|
||
Passing a `str` containing characters that can't be encoded under `client_locale` raises `informix_db.DataError` cleanly — the connection survives:
|
||
|
||
```python
|
||
conn = informix_db.connect(..., client_locale="en_US.8859-1")
|
||
cur = conn.cursor()
|
||
try:
|
||
cur.execute("INSERT INTO t VALUES (?)", ("你好",))
|
||
except informix_db.DataError as e:
|
||
print(e)
|
||
# cannot encode parameter under client_locale codec 'iso-8859-1':
|
||
# ordinal not in range(256) at position 0-2.
|
||
# Connect with a wider locale (e.g., 'en_US.utf8') if your data
|
||
# contains characters outside this codec.
|
||
|
||
# Connection is still good
|
||
cur.execute("SELECT 1 FROM systables WHERE tabid = 1")
|
||
```
|
||
|
||
Protocol-level strings (cursor names, function signatures, error "near tokens", SQL keywords) are always ASCII and stay `iso-8859-1` regardless of `client_locale`.
|
||
|
||
## Cursor lifecycle
|
||
|
||
```python
|
||
cur = conn.cursor()
|
||
cur.execute("SELECT id, name FROM users WHERE active = ?", (True,))
|
||
|
||
# Single row
|
||
row = cur.fetchone() # tuple or None
|
||
|
||
# All rows
|
||
rows = cur.fetchall() # list[tuple]
|
||
|
||
# Bounded batch
|
||
batch = cur.fetchmany(100) # honors cur.arraysize default
|
||
|
||
# Iteration
|
||
for row in cur:
|
||
print(row)
|
||
|
||
cur.close()
|
||
```
|
||
|
||
The connection's `with` block automatically closes both the connection and any open cursors:
|
||
|
||
```python
|
||
with informix_db.connect(...) as conn:
|
||
cur = conn.cursor()
|
||
cur.execute("SELECT 1 FROM systables WHERE tabid = 1")
|
||
print(cur.fetchone())
|
||
# socket closed, cursor torn down
|
||
```
|
||
|
||
## Parameter binding
|
||
|
||
Informix uses `paramstyle = "numeric"` (ESQL/C convention). Both `?` and `:1` / `:2` work:
|
||
|
||
```python
|
||
cur.execute("SELECT id FROM users WHERE name = ? AND age > ?", ("alice", 30))
|
||
|
||
cur.execute(
|
||
"UPDATE users SET email = :2 WHERE id = :1",
|
||
(42, "alice@example.com"),
|
||
)
|
||
```
|
||
|
||
Supported parameter types: `int`, `float`, `str`, `bool`, `None`, `datetime.date`, `datetime.datetime`, `datetime.timedelta`, `decimal.Decimal`, `informix_db.IntervalYM`, `bytes` (BYTE/TEXT params).
|
||
|
||
## Type mapping reference
|
||
|
||
What you put in vs. what comes out:
|
||
|
||
| SQL type | Param accepts | Result returns |
|
||
|---|---|---|
|
||
| `SMALLINT` (16-bit) | `int` (range -32,767..32,767) | `int` |
|
||
| `INT` / `INTEGER` (32-bit) | `int` (range -2³¹+1..2³¹-1) | `int` |
|
||
| `BIGINT` (64-bit) | `int` | `int` |
|
||
| `SERIAL` / `BIGSERIAL` | `int` (omit for auto-assign) | `int` |
|
||
| `SMALLFLOAT` / `REAL` | `float` | `float` |
|
||
| `FLOAT` / `DOUBLE PRECISION` | `float` | `float` |
|
||
| `DECIMAL(p,s)` / `NUMERIC` | `decimal.Decimal` | `decimal.Decimal` |
|
||
| `MONEY(p,s)` | `decimal.Decimal` | `decimal.Decimal` |
|
||
| `CHAR(N)` | `str` (right-trimmed of trailing spaces) | `str` |
|
||
| `VARCHAR(N)` / `NVARCHAR(N)` | `str` | `str` |
|
||
| `NCHAR(N)` | `str` | `str` |
|
||
| `LVARCHAR(N)` | `str` | `str` |
|
||
| `BOOLEAN` | `bool` | `bool` |
|
||
| `DATE` | `datetime.date` | `datetime.date` |
|
||
| `DATETIME YEAR TO DAY` | `datetime.datetime` | `datetime.date` |
|
||
| `DATETIME YEAR TO SECOND` (etc.) | `datetime.datetime` | `datetime.datetime` |
|
||
| `DATETIME HOUR TO SECOND` | not yet | `datetime.time` |
|
||
| `INTERVAL DAY TO FRACTION(5)` | `datetime.timedelta` | `datetime.timedelta` |
|
||
| `INTERVAL YEAR TO MONTH` | `informix_db.IntervalYM` | `informix_db.IntervalYM` |
|
||
| `BYTE` (legacy in-row blob) | `bytes` | `bytes` |
|
||
| `TEXT` (legacy in-row clob) | `bytes` (or `str`, encoded under `conn.encoding`) | `str` |
|
||
| `BLOB` (smart-LOB) | use `cursor.write_blob_column` with `BLOB_PLACEHOLDER` | `informix_db.BlobLocator` (use `cursor.read_blob_column` for bytes) |
|
||
| `CLOB` (smart-LOB) | use `cursor.write_blob_column(..., clob=True)` | `informix_db.ClobLocator` |
|
||
| `ROW(...)` | not yet | `informix_db.RowValue` (raw payload + schema) |
|
||
| `SET(...)` / `MULTISET(...)` / `LIST(...)` | not yet | `informix_db.CollectionValue` |
|
||
| `NULL` (any type) | `None` | `None` |
|
||
|
||
### NULL sentinels
|
||
|
||
Informix encodes NULL inline rather than as a separate flag for fixed-width types:
|
||
|
||
* `INT`: `0x80000000` (`INT_MIN`)
|
||
* `SMALLINT`: `0x8000` (`SHORT_MIN`)
|
||
* `BIGINT`: `0x8000000000000000` (`LONG_MIN`)
|
||
* `REAL` / `FLOAT`: all-`0xff` bytes
|
||
* `DATE`: `0x80000000` (Day_MIN)
|
||
|
||
If your data legitimately contains these values, you'll see them surface as `None` on the Python side. (Real-world usage rarely hits this — `INT_MIN` as a valid value is uncommon — but it's documented behavior, not a bug.)
|
||
|
||
### `IntervalYM`
|
||
|
||
Year-month intervals can't collapse into `datetime.timedelta` because months have variable length. Provided as a small dataclass:
|
||
|
||
```python
|
||
from informix_db import IntervalYM
|
||
|
||
iv = IntervalYM(months=18)
|
||
print(iv.years, iv.remainder_months) # 1 6
|
||
print(str(iv)) # "1-06"
|
||
|
||
cur.execute("INSERT INTO leases (term) VALUES (?)", (iv,))
|
||
```
|
||
|
||
## Transactions
|
||
|
||
Logged-DB transactions are managed implicitly. The driver sends `SQ_BEGIN` before each transaction in non-autocommit mode; `commit()` and `rollback()` close it.
|
||
|
||
```python
|
||
conn = informix_db.connect(..., autocommit=False) # default
|
||
cur = conn.cursor()
|
||
|
||
cur.execute("INSERT INTO orders VALUES (?, ?)", (1, "..."))
|
||
cur.execute("UPDATE inventory SET qty = qty - 1 WHERE sku = ?", ("ABC",))
|
||
conn.commit()
|
||
|
||
cur.execute("INSERT INTO orders VALUES (?, ?)", (2, "..."))
|
||
conn.rollback() # discards the second insert
|
||
```
|
||
|
||
For **unlogged databases**, both `commit()` and `rollback()` are silent no-ops — the connection knows it can't open a transaction (the server returns sqlcode -201 to `SQ_BEGIN`) and caches that state. Same client code works with both DB modes.
|
||
|
||
For **autocommit mode**, each statement commits independently:
|
||
|
||
```python
|
||
conn = informix_db.connect(..., autocommit=True)
|
||
cur = conn.cursor()
|
||
cur.execute("INSERT ...") # already committed
|
||
```
|
||
|
||
## executemany
|
||
|
||
Batched DML — PREPARE once, BIND/EXECUTE per row, RELEASE at the end:
|
||
|
||
```python
|
||
cur.executemany(
|
||
"INSERT INTO log VALUES (?, ?, ?)",
|
||
[
|
||
(1, "info", "started"),
|
||
(2, "info", "loaded config"),
|
||
(3, "warn", "missing optional setting"),
|
||
],
|
||
)
|
||
conn.commit()
|
||
```
|
||
|
||
## Performance tips
|
||
|
||
Three patterns dominate real-world performance. They're all about **batching the right thing**:
|
||
|
||
### 1. Wrap bulk INSERTs in a transaction (53× speedup)
|
||
|
||
Under `autocommit=True`, **every INSERT forces a server-side transaction-log flush**. Under `autocommit=False`, the flush happens once at COMMIT.
|
||
|
||
| Pattern | 1000 rows | Per row | Throughput |
|
||
|---|---|---|---|
|
||
| `executemany` autocommit=True | 1.72 s | 1.72 ms | ~580 rows/sec |
|
||
| `executemany` in single txn | 32 ms | **32 µs** | **~31,000 rows/sec** |
|
||
|
||
```python
|
||
# Slow — every row commits independently
|
||
conn = informix_db.connect(..., autocommit=True)
|
||
conn.cursor().executemany("INSERT ...", rows)
|
||
|
||
# Fast — one log flush at the end
|
||
conn = informix_db.connect(..., autocommit=False) # default
|
||
cur = conn.cursor()
|
||
cur.executemany("INSERT ...", rows)
|
||
conn.commit()
|
||
```
|
||
|
||
This is the single biggest win for any bulk-load workload.
|
||
|
||
### 2. Use `executemany`, not a loop of `execute` (≈100× speedup)
|
||
|
||
`executemany` PREPAREs once and BIND+EXECUTEs per row. A naive loop PREPAREs and RELEASEs per row — paying the server-side parse cost N times.
|
||
|
||
```python
|
||
# Slow: 1.88 ms per row, dominated by PREPARE/RELEASE overhead
|
||
for row in rows:
|
||
cur.execute("INSERT INTO t VALUES (?, ?, ?)", row)
|
||
|
||
# Fast: shares the prepared statement across all rows
|
||
cur.executemany("INSERT INTO t VALUES (?, ?, ?)", rows)
|
||
```
|
||
|
||
### 3. Use a connection pool (72× speedup over cold connect)
|
||
|
||
Cold connect takes ~11 ms (TCP + login handshake). Pool acquire takes ~150 µs. If your application opens a fresh connection per request, fix that first.
|
||
|
||
```python
|
||
# In a long-lived process (FastAPI, Django, worker), open the pool once
|
||
pool = informix_db.create_pool(host="...", min_size=2, max_size=10)
|
||
|
||
# Per request:
|
||
with pool.connection() as conn:
|
||
cur = conn.cursor()
|
||
cur.execute(...)
|
||
```
|
||
|
||
### Other tips
|
||
|
||
* **Cursor reuse is fine across queries** — but each `execute()` resets `description`, `rowcount`, and the materialized result set. If you need the prior query's data, capture it before re-executing.
|
||
* **`fetchall()` materializes the whole result set in memory.** For large queries, iterate (`for row in cur:`) or use `fetchmany(N)`. Internally the cursor still buffers a server-fetch worth of rows at a time.
|
||
* **The `fast_path_call` API is dramatically cheaper than equivalent SQL** for repeated UDF invocations — routine handles are cached per-connection, so the second call onwards skips the `SQ_GETROUTINE` round-trip.
|
||
|
||
For raw numbers (codec speed, round-trip latencies, full bench results), see `tests/benchmarks/README.md`.
|
||
|
||
## Scrollable cursors
|
||
|
||
A regular cursor walks rows forward only via `fetchone` / `fetchmany` / iteration. The **`fetch_*` family** lets you move backwards, jump to absolute positions, fetch the last row directly, and revisit rows already seen.
|
||
|
||
```python
|
||
cur = conn.cursor()
|
||
cur.execute("SELECT id, name FROM users ORDER BY id")
|
||
|
||
# Standard methods still work
|
||
first = cur.fetchone() # row 0
|
||
second = cur.fetchone() # row 1
|
||
|
||
# Plus the scroll surface
|
||
last = cur.fetch_last() # last row
|
||
prev = cur.fetch_prior() # one back from current
|
||
specific = cur.fetch_absolute(50) # row 50 (0-indexed)
|
||
relative = cur.fetch_relative(-3) # 3 rows back from current
|
||
back_to_start = cur.fetch_first() # row 0
|
||
|
||
# PEP 249 scroll()
|
||
cur.scroll(5, mode="relative") # forward 5 from current
|
||
cur.scroll(0, mode="absolute") # to row 0
|
||
|
||
# Where am I?
|
||
print(cur.rownumber) # 0-indexed; None at before-first / after-last
|
||
```
|
||
|
||
### Two modes: in-memory vs server-side
|
||
|
||
The default cursor materializes the full result set into Python memory on `execute`, then `fetch_*` methods operate on the buffer. Random access is essentially free, but memory grows with row count.
|
||
|
||
Pass `scrollable=True` to `cursor()` to get a **server-side** scroll cursor:
|
||
|
||
```python
|
||
cur = conn.cursor(scrollable=True)
|
||
cur.execute("SELECT id, name FROM big_table")
|
||
last_row = cur.fetch_last() # one round-trip, no buffer
|
||
row_500 = cur.fetch_absolute(500) # one round-trip
|
||
```
|
||
|
||
Server-side mode keeps the cursor open on the server and issues a `SQ_SFETCH` round-trip per scroll operation. Constant client memory, network round-trip per move. Use it when your result set is large enough that materializing it would be wasteful.
|
||
|
||
| Mode | When to use |
|
||
|---|---|
|
||
| `cursor()` (default) | Result fits comfortably in memory (~thousands of rows). All `fetch_*` methods are local; fastest random access. |
|
||
| `cursor(scrollable=True)` | Large result sets where memory matters. Each scroll operation is a round-trip; cursor stays open server-side until `close()`. |
|
||
|
||
Server-side scroll cursors require non-autocommit mode (the server needs an open transaction to keep the cursor alive across fetches).
|
||
|
||
### Edge cases
|
||
|
||
* `fetch_prior()` from past-end returns the **last** row (SQL standard semantics — the first prior from "after-last" is the last actual row, not the second-to-last).
|
||
* `fetch_absolute(0)` is the first row; `fetch_absolute(-1)` is the last row (Python-style negative indexing).
|
||
* `cursor.rownumber` is 0-indexed; returns `None` when positioned before-first or after-last, or when no result set exists.
|
||
|
||
## Smart-LOBs (BLOB / CLOB)
|
||
|
||
### Read
|
||
|
||
```python
|
||
# Fetch a single row's BLOB content as bytes
|
||
data = cur.read_blob_column(
|
||
"SELECT data FROM photos WHERE id = ?", (42,)
|
||
)
|
||
# data is bytes (or None if NULL or no rows match)
|
||
```
|
||
|
||
For multi-row reads or full control, drop down to the lower-level
|
||
`lotofile()` SQL form:
|
||
|
||
```python
|
||
cur.execute(
|
||
"SELECT id, lotofile(data, '/tmp/x', 'client') FROM photos LIMIT 100"
|
||
)
|
||
for row in cur:
|
||
photo_id, returned_filename = row
|
||
raw_bytes = cur.blob_files[returned_filename]
|
||
process(photo_id, raw_bytes)
|
||
```
|
||
|
||
The server returns a unique filename suffix for each row; `cur.blob_files` is a dict keyed by those names. Phase 10 in the [decision log](DECISION_LOG.md) explains the protocol.
|
||
|
||
### Write
|
||
|
||
```python
|
||
cur.write_blob_column(
|
||
"INSERT INTO photos VALUES (?, BLOB_PLACEHOLDER)",
|
||
blob_data=jpeg_bytes,
|
||
params=(42,),
|
||
)
|
||
# CLOB column? Pass clob=True so it routes through filetoclob:
|
||
cur.write_blob_column(
|
||
"INSERT INTO docs VALUES (?, BLOB_PLACEHOLDER)",
|
||
blob_data=text.encode("iso-8859-1"),
|
||
params=(1,),
|
||
clob=True,
|
||
)
|
||
```
|
||
|
||
### Why `BLOB_PLACEHOLDER` instead of `?`?
|
||
|
||
Plain `bytes` already maps to BYTE (legacy in-row blobs, type 11) when used as a `?`-parameter. The token approach makes it unambiguous which column receives the smart-LOB. The driver substitutes `BLOB_PLACEHOLDER` with `filetoblob('<sentinel>', 'client')` and registers the bytes for upload via the `SQ_FILE` protocol.
|
||
|
||
## Connection pool
|
||
|
||
```python
|
||
pool = informix_db.create_pool(
|
||
host="...", user="...", password="...",
|
||
database="mydb",
|
||
min_size=1, # pre-opened on construction
|
||
max_size=10, # hard ceiling
|
||
acquire_timeout=30.0, # seconds to wait for a free connection
|
||
)
|
||
|
||
# Acquire / release via context manager (preferred)
|
||
with pool.connection() as conn:
|
||
cur = conn.cursor()
|
||
cur.execute(...)
|
||
# automatically returned to the pool
|
||
|
||
# Or manually
|
||
conn = pool.acquire(timeout=5.0)
|
||
try:
|
||
cur = conn.cursor()
|
||
cur.execute(...)
|
||
finally:
|
||
pool.release(conn)
|
||
|
||
pool.close() # drains idle connections; in-use connections close on their next release
|
||
```
|
||
|
||
The pool sends a trivial `SELECT 1` round-trip before yielding each connection (cheap health check; ~1ms on local network). Dead connections are silently replaced. Connection-related errors (`OperationalError`, `InterfaceError`) raised inside `with pool.connection() as conn:` evict the connection rather than returning it to the pool.
|
||
|
||
## Async (asyncio)
|
||
|
||
```python
|
||
import asyncio
|
||
from informix_db import aio
|
||
|
||
async def main():
|
||
async with await aio.connect(
|
||
host="...", user="...", password="...", database="mydb",
|
||
) as conn:
|
||
cur = await conn.cursor()
|
||
await cur.execute(
|
||
"SELECT id, name FROM users WHERE active = ?", (True,)
|
||
)
|
||
async for row in cur:
|
||
print(row)
|
||
|
||
asyncio.run(main())
|
||
```
|
||
|
||
Async pool:
|
||
|
||
```python
|
||
pool = await aio.create_pool(
|
||
host="...", user="...", password="...", database="mydb",
|
||
min_size=1, max_size=10,
|
||
)
|
||
async with pool.connection() as conn:
|
||
cur = await conn.cursor()
|
||
await cur.execute(...)
|
||
rows = await cur.fetchall()
|
||
await pool.close()
|
||
```
|
||
|
||
The async API mirrors the sync API one-to-one. Each blocking I/O call is offloaded to a worker thread via `asyncio.to_thread` — the event loop never blocks; concurrent queries across an `asyncio.gather` actually run in parallel up to `max_size`.
|
||
|
||
### Cancellation and timeouts
|
||
|
||
Both styles are safe under Phase 27:
|
||
|
||
```python
|
||
# Connection-level — socket-layer timeout, raises OperationalError
|
||
conn = await aio.connect(..., read_timeout=30.0)
|
||
|
||
# Awaitable-level — works because the pool evicts on CancelledError
|
||
# and the per-connection wire lock prevents interleaved I/O
|
||
await asyncio.wait_for(cur.execute(big_query), timeout=30.0)
|
||
```
|
||
|
||
How it works: every wire op acquires the connection's `_wire_lock` (a re-entrant lock). When an awaitable is cancelled, the underlying `to_thread` worker may still be running — but the pool's `release()` waits up to 5 seconds for the lock. If the worker finishes in time, normal release proceeds (with a transaction rollback if needed). If it doesn't, the connection is evicted instead of recycled. The pool never returns a connection that two threads are touching.
|
||
|
||
Pick whichever timeout style fits your code; you don't need to choose for safety reasons.
|
||
|
||
## TLS
|
||
|
||
```python
|
||
import ssl
|
||
|
||
# Production: caller-supplied SSLContext with full verification
|
||
ctx = ssl.create_default_context(cafile="/path/to/ca.pem")
|
||
informix_db.connect(host="db.example.com", port=9089, ..., tls=ctx)
|
||
|
||
# Dev / self-signed certs: tls=True (verification DISABLED)
|
||
informix_db.connect(host="127.0.0.1", port=9089, ..., tls=True)
|
||
```
|
||
|
||
Informix uses dedicated TLS-enabled listener ports (configured server-side in `sqlhosts`) — point `port` at the TLS listener (often `9089`) when `tls` is enabled.
|
||
|
||
## Error handling
|
||
|
||
The exception hierarchy follows PEP 249:
|
||
|
||
```text
|
||
Warning
|
||
Error
|
||
├── InterfaceError
|
||
└── DatabaseError
|
||
├── DataError
|
||
├── OperationalError
|
||
│ ├── PoolClosedError
|
||
│ └── PoolTimeoutError
|
||
├── IntegrityError
|
||
├── InternalError
|
||
├── ProgrammingError
|
||
└── NotSupportedError
|
||
```
|
||
|
||
Server-side SQL errors carry the Informix `sqlcode`, `isamcode`, byte offset, and "near token" attributes:
|
||
|
||
```python
|
||
try:
|
||
cur.execute("INSERT INTO users VALUES (1, 'duplicate-name')")
|
||
except informix_db.IntegrityError as e:
|
||
print(e.sqlcode) # e.g., -239 (duplicate key)
|
||
print(e.isamcode) # e.g., -100
|
||
print(e.near) # e.g., "u_users_name"
|
||
```
|
||
|
||
The exception class is chosen based on the sqlcode (per the catalog in `informix_db/_errcodes.py`):
|
||
|
||
| sqlcode | Exception class |
|
||
|---------|-----------------|
|
||
| -239, -268, -391, etc. | `IntegrityError` |
|
||
| -201, -202, -206, etc. | `ProgrammingError` |
|
||
| -255, -256, -267, etc. | `OperationalError` |
|
||
| -329, -413, -879, etc. | `NotSupportedError` |
|
||
|
||
## Direct stored-procedure invocation (fast-path RPC)
|
||
|
||
For UDFs that aren't callable via plain SQL (`ifx_lo_close`, etc.) or where you want to skip PREPARE → DESCRIBE → EXECUTE overhead:
|
||
|
||
```python
|
||
result = conn.fast_path_call(
|
||
"function informix.ifx_lo_close(integer)", lofd
|
||
)
|
||
# result is a list of return values; here, [0] on success
|
||
```
|
||
|
||
Routine handles are cached per-connection by signature — first call resolves via `SQ_GETROUTINE`, subsequent calls skip that round-trip. UDT parameters (e.g., the 72-byte BLOB locator type) aren't yet supported on the bind side; only scalar params/returns work in the current MVP.
|
||
|
||
## Server-side requirements
|
||
|
||
Informix dev-image setup once-per-instance for the LOB feature set:
|
||
|
||
```bash
|
||
# Inside the container, as the informix user with INFORMIXDIR/INFORMIXSERVER set:
|
||
onspaces -c -b blobspace1 -p /opt/ibm/data/spaces/blobspace.000 -o 0 -s 50000
|
||
onspaces -c -S sbspace1 -p /opt/ibm/data/spaces/sbspace.000 -o 0 -s 50000 -Df "AVG_LO_SIZE=100"
|
||
onmode -wm SBSPACENAME=sbspace1
|
||
onmode -wm LTAPEDEV=/dev/null
|
||
onmode -wm TAPEDEV=/dev/null
|
||
onmode -l
|
||
ontape -s -L 0 -t /dev/null
|
||
```
|
||
|
||
Then create a logged database (required for BYTE/TEXT/BLOB/CLOB):
|
||
|
||
```sql
|
||
CREATE DATABASE mydb WITH LOG;
|
||
```
|
||
|
||
These steps are detailed in the [DECISION_LOG](DECISION_LOG.md) §6.f and §10.
|
||
|
||
## Known limitations
|
||
|
||
Things that don't work yet (and the workaround when one exists):
|
||
|
||
| Limitation | Workaround |
|
||
|---|---|
|
||
| **Named parameters** (`paramstyle="named"` or `dict` parameters) | Use positional `?` / `:1` / `:2`. PEP 249 declares one paramstyle per module. |
|
||
| **Binding `ROW(...)` / `SET / MULTISET / LIST`** as a parameter | Decode side surfaces these as `RowValue` / `CollectionValue`. For *writes*, use SQL projections to build them server-side. |
|
||
| **GSSAPI / Kerberos / LDAP auth** | Username/password (plain or password obfuscation) only. |
|
||
| **Distributed transactions (XA)** | Out of scope for the current driver. |
|
||
| **Bulk-load via COPY** | Use `executemany` inside a transaction (≈31K rows/sec — see Performance tips). |
|
||
| **`executemany` on SELECT** | Loop `execute(select_sql, params)` — `executemany` is DML-only by design. |
|
||
| **Listener failover / sqlhosts groups** | Connect to a specific host:port. Implement failover at the application layer or behind a load balancer. |
|
||
| **DATETIME `HOUR TO FRACTION` as a parameter** | Use `DATETIME YEAR TO SECOND` (full datetime). Read side handles all qualifier ranges. |
|
||
| **`BlobLocator` / `ClobLocator` as a parameter** | The `read_blob_column` / `write_blob_column` cursor methods cover the BLOB / CLOB I/O cases. Direct locator-as-param will follow when there's a real use case. |
|
||
| **UDT-typed parameters / returns in `fast_path_call`** | Scalar params and returns only (INT / SMALLINT / BIGINT / FLOAT / REAL / CHAR / VARCHAR). Complex UDT bind needs the IfxComplexInput protocol layer (~700 lines, deferred). |
|
||
|
||
Things that work but might surprise you:
|
||
|
||
* **`autocommit=True` is opt-in.** PEP 249's default is `autocommit=False`, and that's our default too. Many users coming from `IfxPy` (which defaults to autocommit-on) will find this different — and dramatically faster for bulk loads (see Performance tips).
|
||
* **`commit()` / `rollback()` on an unlogged DB are silent no-ops.** The server returns sqlcode `-201` to `SQ_BEGIN`; the connection caches that and skips the round-trip on subsequent calls. Same client code works against logged and unlogged databases.
|
||
* **`SERIAL` / `BIGSERIAL` columns omitted from INSERT** auto-assign on the server. The auto-assigned value isn't currently exposed via `cursor.lastrowid` (PEP 249 optional surface) — round-trip via `SELECT DBINFO('sqlca.sqlerrd1') FROM systables WHERE tabid=1` if you need it.
|
||
|
||
## Migration from `IfxPy` / legacy `informixdb`
|
||
|
||
The PEP 249 surface is identical — most code Just Works after switching the import:
|
||
|
||
```python
|
||
# Before
|
||
import IfxPyDbi as ifx
|
||
|
||
# After
|
||
import informix_db as ifx
|
||
```
|
||
|
||
Differences worth knowing:
|
||
|
||
| | `IfxPy` / legacy `informixdb` | `informix-db` |
|
||
|---|---|---|
|
||
| **Native deps** | IBM CSDK (`libifsql.so`) | None |
|
||
| **Wheel size** | ~50MB+ (CSDK bundled) | ~50KB |
|
||
| **Connection string** | DSN format | Per-keyword args (`host=`, `user=`, `password=`, `database=`, `server=`) |
|
||
| **paramstyle** | `qmark` | `numeric` (both `?` and `:N` work) |
|
||
| **TLS** | CSDK-managed | Native Python `ssl.SSLContext` |
|
||
| **Async** | Not supported | `informix_db.aio` |
|
||
| **Pool** | External (e.g., SQLAlchemy) | Built-in (`informix_db.create_pool`) |
|
||
| **BLOB API** | `setBytes`/`getBytes` | `cursor.read_blob_column` / `cursor.write_blob_column` with `BLOB_PLACEHOLDER` |
|