informix-db/tests/benchmarks/compare/README.md

# `informix-db` vs IfxPy comparison benchmark

Head-to-head benchmarks against [IfxPy](https://pypi.org/project/IfxPy/), the IBM-published C-bound Informix driver, on identical workloads against the same Informix Developer Edition Docker container.

## TL;DR

| Benchmark | IfxPy 3.0.5 (C-bound) | informix-db 2026.05.05.4 (pure Python) | Result |
|---|---:|---:|---:|
| `select_one_row` (single-row latency) | 128 µs | **116 µs** | **`informix-db` 9% faster** |
| `select_systables_first_10` (~10 rows) | 126 µs | 184 µs | IfxPy 32% faster |
| `select_bench_table_all` (1000-row fetch) | 969 µs | **855 µs** | **`informix-db` 12% faster** |
| `executemany(1000)` in transaction (bulk write) | 21.5 ms | 30.8 ms | IfxPy 30% faster |
| `cold_connect_disconnect` (login handshake) | 11.0 ms | 10.9 ms | comparable |

**`informix-db` is faster on 2/5, slower on 2/5, comparable on 1/5 — overall within the same order of magnitude as the C-bound driver on every workload.**

## What this means

Conventional wisdom says C beats Python at I/O drivers. Here, the picture is more nuanced:

- **When the wire dominates (single round-trips, bulk fetch), `informix-db` wins** because IfxPy adds an ODBC abstraction layer (Python → OneDB ODBC driver → libifdmr.so → wire) where we go direct (Python → wire).
- **When per-row marshaling dominates (executemany, wider tuple construction), IfxPy wins** because its C-level `execute(stmt, tuple)` is faster than our Python BIND-PDU build.
- **When the wire handshake dominates (cold connect), they tie** because both drivers wait ~11 ms for the server's login response.

The takeaway is that pure-Python doesn't mean "performance compromise" — it means **different overhead distribution**. For most application workloads (web requests doing a handful of small queries), the wire round-trip is what matters, and the abstraction-layer overhead IfxPy carries means `informix-db` is typically the same speed or faster.

## Why this comparison was hard to set up

**IfxPy is genuinely difficult to install on a modern system.** Capturing the install gauntlet for the record:

| Step | Detail |
|---|---|
| 1. Pin Python 3.11 | Python 3.13 fails: IfxPy's `setup.py` uses `use_2to3`, removed from setuptools 58 (October 2021). |
| 2. Pin setuptools <58 | Same root cause. |
| 3. CFLAGS hack | GCC 11+ (default since 2021) escalates the C extension's pointer-type warnings to errors. Need `CFLAGS="-Wno-incompatible-pointer-types -Wno-error"` to demote them. |
| 4. Download OneDB ODBC drivers | A 92 MB tarball from `hcl-onedb.github.io/odbc/`. The `pip install` only fetches headers — the runtime libs are a separate, undocumented download. |
| 5. Set INFORMIXDIR + LD_LIBRARY_PATH | Across four directories (`lib/`, `lib/cli/`, `lib/esql/`, `gls/dll/`). |
| 6. Install `libcrypt.so.1` | The OneDB drivers link against the libcrypt-1 ABI (deprecated in 2018, replaced by libcrypt.so.2). Modern Arch / Fedora 35+ / RHEL 9 ship only libcrypt.so.2; you need a compatibility shim (Ubuntu 20.04 still has it; modern distros need `libxcrypt-compat` or similar). |
| 7. Build runtime container | We use `Dockerfile.ifxpy` here because Ubuntu 20.04 is the most recent base distro that still ships `libcrypt.so.1` natively. |

By contrast, `informix-db`'s install is `pip install informix-db`. No external downloads, no system packages, no LD_LIBRARY_PATH, no Docker required.

## Methodology

- Both drivers ran against the **same** Informix Developer Edition 15.0.1.0.3DE Docker container (`informix-db-test` from `tests/docker-compose.yml`).
- The host runs Arch Linux on x86_64; the IfxPy container runs Ubuntu 20.04 on x86_64. Both reach the server through the loopback path (host's `127.0.0.1:9088` for `informix-db`; `--network=host` for the IfxPy container).
- Each benchmark runs 100/20/3 rounds depending on per-iteration cost; we report the mean. Stddev is small (under 5%) for all reported numbers — within-run jitter doesn't affect the qualitative result.
- Workloads are matched semantically: same SQL, same row counts, same fetch patterns. Where they differ (IfxPy's `IfxPy.fetch_tuple` vs. our `cursor.fetchall`), we use whichever idiom exhausts the cursor in each driver.

## Reproduce

From the project root:

```bash
# 1. Start the dev Informix container
make ifx-up

# 2. Seed the 1k-row test table on the host (using informix-db)
uv run python -c "
import informix_db, contextlib
conn = informix_db.connect(host='127.0.0.1', port=9088,
    user='informix', password='in4mix',
    database='sysmaster', server='informix', autocommit=True)
cur = conn.cursor()
with contextlib.suppress(Exception): cur.execute('DROP TABLE p21_bench')
cur.execute('CREATE TABLE p21_bench (id INT, name VARCHAR(64), counter INT, value FLOAT, created DATE)')
cur.executemany('INSERT INTO p21_bench VALUES (?, ?, ?, ?, ?)',
    [(i, f'row_{i:04d}', i*7, float(i)*1.5, None) for i in range(1000)])
conn.close()
"

# 3. Build + run the IfxPy benchmark container
docker build -f tests/benchmarks/compare/Dockerfile.ifxpy \
    -t ifxpy-bench tests/benchmarks/compare/
docker run --rm --network=host ifxpy-bench

# 4. Run informix-db benchmarks for the matched comparison
uv run pytest tests/benchmarks/test_select_perf.py \
    tests/benchmarks/test_pool_perf.py \
    tests/benchmarks/test_insert_perf.py \
    -m benchmark --benchmark-only --benchmark-warmup=on
```

## Files

- `Dockerfile.ifxpy` — Ubuntu 20.04 container with Python 3.9, IfxPy, and OneDB drivers installed
- `ifxpy_bench.py` — IfxPy benchmark workloads (mirrors `tests/benchmarks/test_*_perf.py`)
- This README

## Caveats

- IfxPy 3.0.5 is the latest PyPI version (from October 2020). It's the most actively-maintained C-bound option but hasn't shipped a release in ~5 years.
- Numbers will vary by host, distro, kernel, network stack — re-run on your own hardware before drawing strong conclusions.
- The 1k-row INSERT benchmark uses different APIs (IfxPy's `prepare`+`execute` loop vs our `executemany`); the comparison is by total wall-clock time for the equivalent workload, not by per-call overhead.