Phase 2: SELECT works end-to-end — pure-Python Informix fully reads data

cursor.execute("SELECT 1 FROM systables WHERE tabid = 1")
  cursor.fetchone() == (1,)

To my knowledge, this is the first time a pure-Python implementation
has read data from Informix without wrapping IBM's CSDK or JDBC.

Three breakthroughs in this commit:

1. Login PDU's database field is BROKEN. Passing a database name there
   makes the server reject subsequent SQ_DBOPEN with sqlcode -759
   ("database not available"). JDBC always sends NULL in the login
   PDU's database slot — we now do the same. The user-supplied database
   opens via SQ_DBOPEN in _init_session.

2. Post-login session init dance: SQ_PROTOCOLS (8-byte feature mask
   replayed verbatim from JDBC) → SQ_INFO with INFO_ENV + env vars
   (48-byte PDU replayed verbatim — DBTEMP=/tmp, SUBQCACHESZ=10) →
   SQ_DBOPEN. Without all three steps in this exact order, the server
   silently ignores SELECTs.

3. SQ_DESCRIBE per-column block has 10 fields per column (not the
   simple "name + type" my best-effort parser assumed): fieldIndex,
   columnStartPos, columnType, columnExtendedId, ownerName,
   extendedName, reference, alignment, sourceType, encodedLength.
   The string table at the end is offset-indexed (fieldIndex points
   into it), which is how JDBC handles disambiguation.

Cursor lifecycle implementation in cursors.py mirrors JDBC exactly:
  PREPARE+NDESCRIBE+WANTDONE → DESCRIBE+DONE+COST+EOT
  CURNAME+NFETCH(4096) → TUPLE*+DONE+COST+EOT
  NFETCH(4096) → DONE+COST+EOT (drain)
  CLOSE → EOT
  RELEASE → EOT

Five round trips per SELECT — same as JDBC.

Module changes:
  src/informix_db/connections.py — added _init_session(), _send_protocols(),
    _send_dbopen(), _drain_to_eot(), _raise_sq_err(); login PDU now
    forces database=None always; SQ_INFO PDU replayed verbatim from
    JDBC capture (offsets-indexed env-var format too gnarly to derive
    in MVP).
  src/informix_db/cursors.py — full rewrite: real PDU builders for
    PREPARE/CURNAME+NFETCH/NFETCH/CLOSE/RELEASE; tag-dispatched
    response readers; cursor-name generator matching JDBC's "_ifxc"
    convention.
  src/informix_db/_resultset.py — proper SQ_DESCRIBE parser per
    JDBC's receiveDescribe (USVER mode); offset-indexed string table
    with name lookup by fieldIndex; ColumnInfo dataclass with raw
    type-code preserved for null-flag extraction.
  src/informix_db/_messages.py — added SQ_NDESCRIBE=22, SQ_WANTDONE=49.

Test coverage: 40 unit + 15 integration tests (7 smoke + 8 new SELECT)
= 55 total, all green, ruff clean. New tests cover:
  - SELECT 1 returns (1,)
  - cursor.description shape per PEP 249
  - Multi-row INT SELECT
  - Multi-column mixed types (INT + FLOAT)
  - Iterator protocol (for row in cursor)
  - fetchmany(n)
  - Re-executing on same cursor resets state
  - Two cursors on one connection (sequential)

Known gap: VARCHAR row decoding doesn't yet handle the variable-width
on-wire encoding correctly. Phase 2.x will address — for now NotImpl
errors surface raw bytes in the row tuple.
This commit is contained in:
Ryan Malloy 2026-05-03 15:37:10 -06:00
parent e2c48f855e
commit a1bd52788d
14 changed files with 742 additions and 228 deletions

View File

@ -0,0 +1,18 @@
2026/05/02 22:50:01 socat[111202] N listening on AF=2 0.0.0.0:9090
2026/05/02 22:50:02 socat[111202] N accepting connection from AF=2 127.0.0.1:55808 on AF=2 127.0.0.1:9090
2026/05/02 22:50:02 socat[111202] N opening connection to 127.0.0.1:9088
2026/05/02 22:50:02 socat[111202] N opening connection to AF=2 127.0.0.1:9088
2026/05/02 22:50:02 socat[111202] N successfully connected from local address AF=2 127.0.0.1:57708
2026/05/02 22:50:02 socat[111202] N successfully connected to 127.0.0.1:9088
2026/05/02 22:50:02 socat[111202] N starting data transfer loop with FDs [6,6] and [5,5]
> 2026/05/02 22:50:02.286297 length=394 from=0 to=393
01 8a 01 3c 00 00 00 64 00 65 00 00 00 3d 00 06 49 45 45 45 4d 00 00 6c 73 71 6c 65 78 65 63 00 00 00 00 00 00 06 39 2e 32 38 30 00 00 0c 52 44 53 23 52 30 30 30 30 30 30 00 00 05 73 71 6c 69 00 00 00 01 3c 00 00 00 00 00 00 00 00 00 01 00 09 69 6e 66 6f 72 6d 69 78 00 00 07 69 6e 34 6d 69 78 00 6f 6c 00 00 00 00 00 00 00 00 00 3d 74 6c 69 74 63 70 00 00 00 00 00 01 00 68 00 0b 00 00 00 03 00 09 69 6e 66 6f 72 6d 69 78 00 00 0a 73 79 73 6d 61 73 74 65 72 00 00 00 00 00 00 00 00 00 00 6a 00 06 00 07 44 42 50 41 54 48 00 00 02 2e 00 00 0e 43 4c 49 45 4e 54 5f 4c 4f 43 41 4c 45 00 00 0d 65 6e 5f 55 53 2e 38 38 35 39 2d 31 00 00 11 43 4c 4e 54 5f 50 41 4d 5f 43 41 50 41 42 4c 45 00 00 02 31 00 00 07 44 42 44 41 54 45 00 00 06 59 34 4d 44 2d 00 00 0c 49 46 58 5f 55 50 44 44 45 53 43 00 00 02 31 00 00 09 4e 4f 44 45 46 44 41 43 00 00 03 6e 6f 00 00 6b 00 00 00 00 00 01 b2 6d 00 00 00 00 00 0b 72 70 6d 2d 62 75 6c 6c 65 74 00 00 00 00 29 2f 68 6f 6d 65 2f 72 70 6d 2f 63 6c 61 75 64 65 2f 69 6e 66 6f 72 6d 69 78 2f 70 79 74 68 6f 6e 2d 6c 69 62 72 61 72 79 00 00 74 00 20 00 00 00 00 00 00 00 00 00 16 69 6e 66 6f 72 6d 69 78 2d 64 62 40 70 69 64 31 31 31 32 31 33 00 00 7f
< 2026/05/02 22:50:02.298394 length=276 from=0 to=275
01 14 02 3c 10 00 00 64 00 65 00 00 00 3d 00 06 49 45 45 45 49 00 00 6c 73 72 76 69 6e 66 78 00 00 00 00 00 00 2f 49 42 4d 20 49 6e 66 6f 72 6d 69 78 20 44 79 6e 61 6d 69 63 20 53 65 72 76 65 72 20 56 65 72 73 69 6f 6e 20 31 35 2e 30 2e 31 2e 30 2e 33 00 00 07 73 65 72 69 61 6c 00 00 09 69 6e 66 6f 72 6d 69 78 00 00 00 01 3c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 6f 6e 00 00 00 00 00 00 00 00 00 3d 73 6f 63 74 63 70 00 00 00 00 00 00 00 66 00 00 00 00 00 00 00 00 00 00 00 15 00 00 00 6b 00 00 00 00 00 00 03 1a 00 00 00 00 00 0d 32 33 32 37 63 34 33 35 34 65 61 38 00 00 00 00 0f 2f 68 6f 6d 65 2f 69 6e 66 6f 72 6d 69 78 00 00 6e 00 04 00 00 00 00 00 74 00 33 00 00 00 c8 00 00 00 c8 00 29 2f 6f 70 74 2f 69 62 6d 2f 69 6e 66 6f 72 6d 69 78 2f 76 31 35 2e 30 2e 31 2e 30 2e 33 2f 62 69 6e 2f 6f 6e 69 6e 69 74 00 00 7f
> 2026/05/02 22:50:02.298951 length=14 from=394 to=407
00 7e 00 08 ff fc 7f fc 3c 8c aa 97 00 0c
< 2026/05/02 22:50:02.299061 length=16 from=276 to=291
00 7e 00 09 bd be 9f fe 7f b7 ff ef ff 00 00 0c
2026/05/02 22:50:05 socat[111202] N socket 1 (fd 6) is at EOF
2026/05/02 22:50:05 socat[111202] N socket 2 (fd 5) is at EOF
2026/05/02 22:50:05 socat[111202] N exiting with status 0

View File

@ -0,0 +1,20 @@
2026/05/02 22:51:21 socat[114183] N listening on AF=2 0.0.0.0:9090
2026/05/02 22:51:22 socat[114183] N accepting connection from AF=2 127.0.0.1:53132 on AF=2 127.0.0.1:9090
2026/05/02 22:51:22 socat[114183] N opening connection to 127.0.0.1:9088
2026/05/02 22:51:22 socat[114183] N opening connection to AF=2 127.0.0.1:9088
2026/05/02 22:51:22 socat[114183] N successfully connected from local address AF=2 127.0.0.1:37058
2026/05/02 22:51:22 socat[114183] N successfully connected to 127.0.0.1:9088
2026/05/02 22:51:22 socat[114183] N starting data transfer loop with FDs [6,6] and [5,5]
> 2026/05/02 22:51:22.136752 length=394 from=0 to=393
01 8a 01 3c 00 00 00 64 00 65 00 00 00 3d 00 06 49 45 45 45 4d 00 00 6c 73 71 6c 65 78 65 63 00 00 00 00 00 00 06 39 2e 32 38 30 00 00 0c 52 44 53 23 52 30 30 30 30 30 30 00 00 05 73 71 6c 69 00 00 00 01 3c 00 00 00 00 00 00 00 00 00 01 00 09 69 6e 66 6f 72 6d 69 78 00 00 07 69 6e 34 6d 69 78 00 6f 6c 00 00 00 00 00 00 00 00 00 3d 74 6c 69 74 63 70 00 00 00 00 00 01 00 68 00 0b 00 00 00 03 00 09 69 6e 66 6f 72 6d 69 78 00 00 0a 73 79 73 6d 61 73 74 65 72 00 00 00 00 00 00 00 00 00 00 6a 00 06 00 07 44 42 50 41 54 48 00 00 02 2e 00 00 0e 43 4c 49 45 4e 54 5f 4c 4f 43 41 4c 45 00 00 0d 65 6e 5f 55 53 2e 38 38 35 39 2d 31 00 00 11 43 4c 4e 54 5f 50 41 4d 5f 43 41 50 41 42 4c 45 00 00 02 31 00 00 07 44 42 44 41 54 45 00 00 06 59 34 4d 44 2d 00 00 0c 49 46 58 5f 55 50 44 44 45 53 43 00 00 02 31 00 00 09 4e 4f 44 45 46 44 41 43 00 00 03 6e 6f 00 00 6b 00 00 00 00 00 01 be 13 00 00 00 00 00 0b 72 70 6d 2d 62 75 6c 6c 65 74 00 00 00 00 29 2f 68 6f 6d 65 2f 72 70 6d 2f 63 6c 61 75 64 65 2f 69 6e 66 6f 72 6d 69 78 2f 70 79 74 68 6f 6e 2d 6c 69 62 72 61 72 79 00 00 74 00 20 00 00 00 00 00 00 00 00 00 16 69 6e 66 6f 72 6d 69 78 2d 64 62 40 70 69 64 31 31 34 31 39 35 00 00 7f
< 2026/05/02 22:51:22.148900 length=276 from=0 to=275
01 14 02 3c 10 00 00 64 00 65 00 00 00 3d 00 06 49 45 45 45 49 00 00 6c 73 72 76 69 6e 66 78 00 00 00 00 00 00 2f 49 42 4d 20 49 6e 66 6f 72 6d 69 78 20 44 79 6e 61 6d 69 63 20 53 65 72 76 65 72 20 56 65 72 73 69 6f 6e 20 31 35 2e 30 2e 31 2e 30 2e 33 00 00 07 73 65 72 69 61 6c 00 00 09 69 6e 66 6f 72 6d 69 78 00 00 00 01 3c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 6f 6e 00 00 00 00 00 00 00 00 00 3d 73 6f 63 74 63 70 00 00 00 00 00 00 00 66 00 00 00 00 00 00 00 00 00 00 00 15 00 00 00 6b 00 00 00 00 00 00 03 1a 00 00 00 00 00 0d 32 33 32 37 63 34 33 35 34 65 61 38 00 00 00 00 0f 2f 68 6f 6d 65 2f 69 6e 66 6f 72 6d 69 78 00 00 6e 00 04 00 00 00 00 00 74 00 33 00 00 00 c8 00 00 00 c8 00 29 2f 6f 70 74 2f 69 62 6d 2f 69 6e 66 6f 72 6d 69 78 2f 76 31 35 2e 30 2e 31 2e 30 2e 33 2f 62 69 6e 2f 6f 6e 69 6e 69 74 00 00 7f
> 2026/05/02 22:51:22.149470 length=14 from=394 to=407
00 7e 00 08 ff fc 7f fc 3c 8c aa 97 00 0c
< 2026/05/02 22:51:22.149602 length=16 from=276 to=291
00 7e 00 09 bd be 9f fe 7f b7 ff ef ff 00 00 0c
> 2026/05/02 22:51:22.149652 length=8 from=408 to=415
00 51 00 06 00 26 00 0c
2026/05/02 22:51:24 socat[114183] N socket 1 (fd 6) is at EOF
2026/05/02 22:51:24 socat[114183] N socket 2 (fd 5) is at EOF
2026/05/02 22:51:24 socat[114183] N exiting with status 0

View File

@ -0,0 +1,26 @@
2026/05/03 15:29:01 socat[2331545] N listening on AF=2 0.0.0.0:9090
2026/05/03 15:29:01 socat[2331545] N accepting connection from AF=2 127.0.0.1:59658 on AF=2 127.0.0.1:9090
2026/05/03 15:29:01 socat[2331545] N opening connection to 127.0.0.1:9088
2026/05/03 15:29:01 socat[2331545] N opening connection to AF=2 127.0.0.1:9088
2026/05/03 15:29:01 socat[2331545] N successfully connected from local address AF=2 127.0.0.1:35104
2026/05/03 15:29:01 socat[2331545] N successfully connected to 127.0.0.1:9088
2026/05/03 15:29:01 socat[2331545] N starting data transfer loop with FDs [6,6] and [5,5]
> 2026/05/03 15:29:01.775258 length=395 from=0 to=394
01 8b 01 3c 00 00 00 64 00 65 00 00 00 3d 00 06 49 45 45 45 4d 00 00 6c 73 71 6c 65 78 65 63 00 00 00 00 00 00 06 39 2e 32 38 30 00 00 0c 52 44 53 23 52 30 30 30 30 30 30 00 00 05 73 71 6c 69 00 00 00 01 3c 00 00 00 00 00 00 00 00 00 01 00 09 69 6e 66 6f 72 6d 69 78 00 00 07 69 6e 34 6d 69 78 00 6f 6c 00 00 00 00 00 00 00 00 00 3d 74 6c 69 74 63 70 00 00 00 00 00 01 00 68 00 0b 00 00 00 03 00 09 69 6e 66 6f 72 6d 69 78 00 00 0a 73 79 73 6d 61 73 74 65 72 00 00 00 00 00 00 00 00 00 00 6a 00 06 00 07 44 42 50 41 54 48 00 00 02 2e 00 00 0e 43 4c 49 45 4e 54 5f 4c 4f 43 41 4c 45 00 00 0d 65 6e 5f 55 53 2e 38 38 35 39 2d 31 00 00 11 43 4c 4e 54 5f 50 41 4d 5f 43 41 50 41 42 4c 45 00 00 02 31 00 00 07 44 42 44 41 54 45 00 00 06 59 34 4d 44 2d 00 00 0c 49 46 58 5f 55 50 44 44 45 53 43 00 00 02 31 00 00 09 4e 4f 44 45 46 44 41 43 00 00 03 6e 6f 00 00 6b 00 00 00 00 00 23 93 c8 00 00 00 00 00 0b 72 70 6d 2d 62 75 6c 6c 65 74 00 00 00 00 29 2f 68 6f 6d 65 2f 72 70 6d 2f 63 6c 61 75 64 65 2f 69 6e 66 6f 72 6d 69 78 2f 70 79 74 68 6f 6e 2d 6c 69 62 72 61 72 79 00 00 74 00 21 00 00 00 00 00 00 00 00 00 17 69 6e 66 6f 72 6d 69 78 2d 64 62 40 70 69 64 32 33 33 31 35 39 32 00 00 7f
< 2026/05/03 15:29:01.787211 length=276 from=0 to=275
01 14 02 3c 10 00 00 64 00 65 00 00 00 3d 00 06 49 45 45 45 49 00 00 6c 73 72 76 69 6e 66 78 00 00 00 00 00 00 2f 49 42 4d 20 49 6e 66 6f 72 6d 69 78 20 44 79 6e 61 6d 69 63 20 53 65 72 76 65 72 20 56 65 72 73 69 6f 6e 20 31 35 2e 30 2e 31 2e 30 2e 33 00 00 07 73 65 72 69 61 6c 00 00 09 69 6e 66 6f 72 6d 69 78 00 00 00 01 3c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 6f 6e 00 00 00 00 00 00 00 00 00 3d 73 6f 63 74 63 70 00 00 00 00 00 00 00 66 00 00 00 00 00 00 00 00 00 00 00 15 00 00 00 6b 00 00 00 00 00 00 03 1a 00 00 00 00 00 0d 32 33 32 37 63 34 33 35 34 65 61 38 00 00 00 00 0f 2f 68 6f 6d 65 2f 69 6e 66 6f 72 6d 69 78 00 00 6e 00 04 00 00 00 00 00 74 00 33 00 00 00 c8 00 00 00 c8 00 29 2f 6f 70 74 2f 69 62 6d 2f 69 6e 66 6f 72 6d 69 78 2f 76 31 35 2e 30 2e 31 2e 30 2e 33 2f 62 69 6e 2f 6f 6e 69 6e 69 74 00 00 7f
> 2026/05/03 15:29:01.787443 length=14 from=395 to=408
00 7e 00 08 ff fc 7f fc 3c 8c aa 97 00 0c
< 2026/05/03 15:29:01.787516 length=16 from=276 to=291
00 7e 00 09 bd be 9f fe 7f b7 ff ef ff 00 00 0c
> 2026/05/03 15:29:01.787565 length=48 from=409 to=456
00 51 00 06 00 26 00 0c 00 04 00 06 44 42 54 45 4d 50 00 04 2f 74 6d 70 00 0b 53 55 42 51 43 41 43 48 45 53 5a 00 00 02 31 30 00 00 00 00 00 0c
< 2026/05/03 15:29:01.787639 length=2 from=292 to=293
00 0c
> 2026/05/03 15:29:01.787658 length=18 from=457 to=474
00 24 00 09 73 79 73 6d 61 73 74 65 72 00 00 00 00 0c
< 2026/05/03 15:29:01.787695 length=14 from=294 to=307
00 0d fd 09 00 00 00 00 00 00 00 00 00 0c
2026/05/03 15:29:01 socat[2331545] N socket 1 (fd 6) is at EOF
2026/05/03 15:29:01 socat[2331545] N socket 2 (fd 5) is at EOF
2026/05/03 15:29:01 socat[2331545] N exiting with status 0

View File

@ -94,11 +94,16 @@ def connect(
the server still requires a successful login for this to work.
"""
return Connection(
host=host, port=port,
user=user, password=password,
database=database, server=server,
connect_timeout=connect_timeout, read_timeout=read_timeout,
host=host,
port=port,
user=user,
password=password,
database=database,
server=server,
connect_timeout=connect_timeout,
read_timeout=read_timeout,
keepalive=keepalive,
client_locale=client_locale, env=env,
client_locale=client_locale,
env=env,
autocommit=autocommit,
)

View File

@ -34,7 +34,7 @@ class MessageType(IntEnum):
SQ_CLOSE = 10
SQ_RELEASE = 11
SQ_NDESCRIBE = 22 # numerical describe — request column metadata after a PREPARE/COMMAND
SQ_WANTDONE = 49 # request a SQ_DONE completion notification
SQ_WANTDONE = 49 # request a SQ_DONE completion notification
# --- Per-PDU framing ---
SQ_EOT = 12 # end-of-transmission / flush marker; ends every PDU

View File

@ -144,9 +144,7 @@ class IfxStreamReader:
while remaining > 0:
chunk = self._source.read(remaining)
if not chunk:
raise ProtocolError(
f"unexpected EOF: wanted {n} bytes, got {n - remaining}"
)
raise ProtocolError(f"unexpected EOF: wanted {n} bytes, got {n - remaining}")
chunks.append(chunk)
remaining -= len(chunk)
return b"".join(chunks)

View File

@ -1,11 +1,21 @@
"""SQ_DESCRIBE column descriptor parser and SQ_TUPLE row decoder.
Best-effort initial implementation derived from
``com.informix.jdbc.IfxSqli.receiveDescribe`` and ``receiveTuple``
(see ``docs/JDBC_NOTES.md``). The exact byte layout of the per-column
descriptor block is intricate enough that we iterate against live
captures and the running container; this module is the integration
point for that iteration.
Per IfxSqli.receiveDescribe (line 2175+) for ``isUSVER`` modern servers.
The per-field block layout is:
fieldIndex (int 4)
columnStartPos (int 4 USVER)
columnType (short 2 base IDS type code with high-bit flags)
columnExtendedId (int 4 USVER, for UDT/extended types)
ownerName (readChar = [short len][bytes][pad if odd])
extendedName (readChar)
reference (short 2)
alignment (short 2)
sourceType (int 4)
encodedLength (int 4)
After all fields: the string table (a length-prefixed block of nul-separated
column names), read via readPadded.
"""
from __future__ import annotations
@ -13,58 +23,64 @@ from __future__ import annotations
from dataclasses import dataclass
from ._protocol import IfxStreamReader
from ._types import IfxType, base_type, is_nullable
from ._types import base_type, is_nullable
from .converters import FIXED_WIDTHS, decode
@dataclass
class ColumnInfo:
"""One column in a SQ_DESCRIBE response.
Maps to PEP 249's ``cursor.description`` row format:
``(name, type_code, display_size, internal_size, precision, scale, null_ok)``.
"""
"""One column in a SQ_DESCRIBE response."""
name: str
type_code: int # base IDS type code (high-bit flags stripped)
flags: int # raw flags byte (NOTNULLABLE etc.)
length: int # declared size on the wire
precision: int = 0
scale: int = 0
raw_type_code: int # raw type-code short with flags intact
encoded_length: int
column_start_pos: int = 0
extended_id: int = 0
owner_name: str = ""
extended_name: str = ""
@property
def null_ok(self) -> bool:
return is_nullable(self.flags << 8 | self.type_code)
return is_nullable(self.raw_type_code)
def to_description_tuple(self) -> tuple:
"""Build the 7-tuple PEP 249 cursor.description expects."""
"""The PEP 249 cursor.description 7-tuple."""
return (
self.name,
self.type_code,
self.length, # display_size
self.length, # internal_size
self.precision,
self.scale,
self.encoded_length, # display_size
self.encoded_length, # internal_size
0, # precision (Phase 6+ derives from type)
0, # scale
self.null_ok,
)
def _read_char(reader: IfxStreamReader, encoding: str = "iso-8859-1") -> str:
"""Read JDBC's ``readChar`` format: [short len][bytes][pad if odd-len]."""
length = reader.read_short()
if length < 0:
return ""
if length == 0:
return ""
data = reader.read_exact(length)
if length & 1:
reader.read_exact(1) # pad byte
return data.decode(encoding)
def parse_describe(reader: IfxStreamReader) -> tuple[list[ColumnInfo], dict]:
"""Parse a SQ_DESCRIBE response payload (the SQ_DESCRIBE tag is already consumed).
"""Parse a SQ_DESCRIBE response (the SQ_DESCRIBE tag is already consumed).
Returns ``(columns, metadata)`` where ``metadata`` carries the
statement-level fields (statementType, statementID, estimatedCost,
tupleSize) for diagnostics.
This is a best-effort initial parser per JDBC's ``receiveDescribe``.
Field-index handling assumes 4-byte offsets (modern servers).
Returns ``(columns, metadata)``.
"""
statement_type = reader.read_short()
statement_id = reader.read_short()
estimated_cost = reader.read_int()
tuple_size = reader.read_short()
nfields = reader.read_short()
string_table_size = reader.read_int() # assume is4ByteOffsetSupported
string_table_size = reader.read_int() # 4-byte on modern servers
metadata = {
"statement_type": statement_type,
@ -78,48 +94,67 @@ def parse_describe(reader: IfxStreamReader) -> tuple[list[ColumnInfo], dict]:
if nfields <= 0:
return [], metadata
# Per-column descriptors. The exact layout per JDBC's loop:
# for each field:
# field_index (int — offset into string table where name lives)
# if isUSVER: column_start_pos (int)
# ... more per-column data ...
#
# For Phase 2 MVP we read defensively: collect raw field offsets,
# then parse the string table at the end. The per-column type/length
# data layout needs more characterization — for now we read the
# field_index and rely on a heuristic walk of the remaining bytes.
field_indexes = []
# Pass 1: per-field descriptor block (no name yet — names come from
# the string table).
raw_fields: list[dict] = []
for _ in range(nfields):
field_indexes.append(reader.read_int())
field_index = reader.read_int()
column_start_pos = reader.read_int()
column_type = reader.read_short()
column_extended_id = reader.read_int()
owner_name = _read_char(reader)
extended_name = _read_char(reader)
reference = reader.read_short() # noqa: F841 (Phase 6+)
alignment = reader.read_short() # noqa: F841
source_type = reader.read_int() # noqa: F841
encoded_length = reader.read_int()
raw_fields.append(
{
"field_index": field_index,
"column_start_pos": column_start_pos,
"type_code": column_type,
"extended_id": column_extended_id,
"owner_name": owner_name,
"extended_name": extended_name,
"encoded_length": encoded_length,
}
)
# Pass 2: string table — nul-separated column names. readPadded.
string_table = b""
if string_table_size > 0:
string_table = reader.read_exact(string_table_size)
if string_table_size & 1:
reader.read_exact(1) # pad
# Split string table on nul to get the column-name list. The fieldIndex
# values point into this table for each column's name.
raw_names = string_table.split(b"\x00")
name_lookup = {0: ""}
cursor = 0
for piece in raw_names:
if piece:
name_lookup[cursor] = piece.decode("iso-8859-1")
cursor += len(piece) + 1 # +1 for the nul we split on
# The remaining bytes hold per-column type info + the string table
# (column names). We don't have the full layout decoded yet, so for
# Phase 2 MVP we extract column names from what looks like the
# length-prefixed name block at the end.
columns: list[ColumnInfo] = []
# Heuristic: skip ahead until we hit a printable ASCII byte that
# plausibly starts a column name. This is wrong in general but works
# for the SELECT 1 case where the only column is "(constant)".
# Phase 2.1 replaces this with a real parser.
raw_remaining = reader.read_exact(string_table_size)
# The string table contains nul-terminated column names back-to-back.
parts = raw_remaining.split(b"\x00")
names = [p.decode("iso-8859-1") for p in parts if p]
# Pad/truncate to nfields
while len(names) < nfields:
names.append(f"col{len(names)}")
names = names[:nfields]
# We don't yet know the per-column type code from the descriptor block.
# For Phase 2 MVP we infer from the SQ_TUPLE payload size when it
# arrives — see decode_tuple. Leave type_code=IfxType.INT as default.
for name in names:
for fd in raw_fields:
# fieldIndex is the byte offset where the column's name starts.
name = name_lookup.get(fd["field_index"])
if name is None:
# Walk the string table to find the name at this offset.
tail = string_table[fd["field_index"] :].split(b"\x00", 1)[0]
name = tail.decode("iso-8859-1") if tail else f"col{len(columns)}"
columns.append(
ColumnInfo(
name=name,
type_code=int(IfxType.INT), # MVP placeholder — refined in Phase 2.1
flags=0,
length=4,
name=name or f"col{len(columns)}",
type_code=base_type(fd["type_code"]),
raw_type_code=fd["type_code"],
encoded_length=fd["encoded_length"],
column_start_pos=fd["column_start_pos"],
extended_id=fd["extended_id"],
owner_name=fd["owner_name"],
extended_name=fd["extended_name"],
)
)
return columns, metadata
@ -131,14 +166,10 @@ def parse_tuple_payload(
) -> tuple:
"""Parse a SQ_TUPLE payload (the SQ_TUPLE tag is already consumed).
JDBC's ``receiveTuple`` reads:
[short warn] [int size] [bytes payload]
The payload is then split into per-column values by walking the
column descriptors. For Phase 2 MVP with fixed-width types only,
each column consumes ``FIXED_WIDTHS[type_code]`` bytes.
Per ``IfxSqli.receiveTuple``:
``[short warn][int size][bytes payload]``
"""
warn = reader.read_short() # noqa: F841 — diagnostic, not surfaced yet
reader.read_short() # warn (Phase 5 surfaces)
size = reader.read_int()
payload = reader.read_exact(size)
@ -148,13 +179,15 @@ def parse_tuple_payload(
base = base_type(col.type_code)
width = FIXED_WIDTHS.get(base)
if width is None:
# Variable-width type — Phase 2.1 work
raise NotImplementedError(
f"variable-width column type {base} not yet supported "
f"(column {col.name!r})"
)
raw = payload[offset:offset + width]
# Variable-width: use encoded_length from the descriptor for now.
# Phase 2.x adds per-type variable-width parsing (e.g. CHAR uses
# encoded_length, VARCHAR has a length prefix in the payload).
width = col.encoded_length
raw = payload[offset : offset + width]
offset += width
values.append(decode(col.type_code, raw))
try:
values.append(decode(col.type_code, raw))
except NotImplementedError:
# Best-effort: surface the raw bytes for unsupported types
values.append(raw)
return tuple(values)

View File

@ -36,9 +36,7 @@ class IfxSocket:
try:
sock = socket.create_connection((host, port), timeout=connect_timeout)
except OSError as e:
raise OperationalError(
f"cannot connect to Informix at {host}:{port}: {e}"
) from e
raise OperationalError(f"cannot connect to Informix at {host}:{port}: {e}") from e
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
if keepalive:
@ -80,8 +78,7 @@ class IfxSocket:
if not chunk:
self._force_close()
raise OperationalError(
f"server closed connection mid-read "
f"(wanted {n} bytes, got {n - remaining})"
f"server closed connection mid-read (wanted {n} bytes, got {n - remaining})"
)
chunks.append(chunk)
remaining -= len(chunk)

View File

@ -104,16 +104,26 @@ class Connection:
self._env.update(env)
self._sock = IfxSocket(
host, port,
host,
port,
connect_timeout=connect_timeout,
read_timeout=read_timeout,
keepalive=keepalive,
)
try:
login_pdu = self._build_login_pdu(user, password)
# The login PDU's database field is BROKEN — passing a db name
# there makes the server reject subsequent SQ_DBOPEN with
# sqlcode=-759. JDBC always sends NULL in the login PDU's database
# slot and then opens the db via SQ_DBOPEN in the post-login init.
# We do the same. The actual database opens in `_init_session`.
login_pdu = self._build_login_pdu(user, password, login_database=None)
self._sock.write_all(login_pdu)
self._parse_login_response()
# Post-login session init: protocol negotiation + (optional) DBOPEN.
# Without this, the server silently drops PREPAREs and rejects DBOPEN
# — see PROTOCOL_NOTES.md §6c for the discovery story.
self._init_session()
except Exception:
self._sock.close()
self._closed = True
@ -164,9 +174,151 @@ class Connection:
def __exit__(self, *_exc: object) -> None:
self.close()
# -- post-login session init ------------------------------------------
def _init_session(self) -> None:
"""Run the post-login session init dance.
After login the server is in a 'connected but not initialized' state.
Before any SELECT/DML works, it needs:
1. ``SQ_PROTOCOLS`` feature-bitmap negotiation (the server ignores
PREPAREs until this completes)
2. ``SQ_DBOPEN`` explicit database open (the login PDU's database
field is advisory only; without DBOPEN the server returns
sqlcode -759 on queries)
We skip the JDBC-additional SQ_INFO and SQ_ID(env vars) steps for
now they don't appear strictly required for the basic SELECT path.
Phase 2.x can re-add them if needed.
"""
# Step 1: SQ_PROTOCOLS — feature-bitmap negotiation.
# The 8-byte protocols mask is the JDBC reference value from
# docs/CAPTURES/02-select-1.socat.log; we replay it verbatim
# since the bits are opaque (server-recognized features).
protocols_mask = bytes.fromhex("fffc7ffc3c8caa97")
self._send_protocols(protocols_mask)
self._drain_to_eot()
# Step 2: SQ_INFO with INFO_ENV subtype + session env vars.
# The actual on-wire format (from JDBC's sendEnv at IfxSqli.java
# line 2990) is:
# [short SQ_INFO=81][short INFO_ENV=6][short totLen]
# [short LongNameLen][short LongValueLen]
# [for each env var: writeChar(name); writeChar(value)]
# [short 0][short 0] # INFO_DONE markers
# [short SQ_EOT=12]
# Where each writeChar emits [short length][bytes][optional pad].
#
# We replay JDBC's exact 48-byte PDU verbatim. Decoded structure:
# 00 51 00 06 00 26 SQ_INFO + INFO_ENV + totLen=38
# 00 0c 00 04 LongNameLen=12, LongValueLen=4
# 00 06 "DBTEMP" nameLen=6, "DBTEMP" (even, no pad)
# 00 04 "/tmp" valueLen=4, "/tmp" (even, no pad)
# 00 0b "SUBQCACHESZ" 00 nameLen=11, name + 1-byte pad
# 00 02 "10" valueLen=2, "10" (even, no pad)
# 00 00 00 00 INFO_DONE markers (two short 0s)
# 00 0c SQ_EOT
# Hex extracted directly from docs/CAPTURES/02-select-1.socat.log.
self._sock.write_all(
bytes.fromhex(
"005100060026000c00040006444254454d5000042f746d70"
"000b535542514341434845535a000002313000000000000c"
)
)
self._drain_to_eot()
# Step 3: SQ_DBOPEN, if the user requested a specific database.
if self._database is not None:
self._send_dbopen(self._database)
self._drain_to_eot()
def _send_protocols(self, protocols: bytes) -> None:
"""Emit a SQ_PROTOCOLS PDU per ``IfxSqli.sendProtocols``.
Layout: ``[short SQ_PROTOCOLS=126][short payloadLen][bytes payload pad-even][short SQ_EOT]``
"""
writer, buf = make_pdu_writer()
writer.write_short(MessageType.SQ_PROTOCOLS)
writer.write_short(len(protocols))
writer.write_padded(protocols)
writer.write_short(MessageType.SQ_EOT)
self._sock.write_all(buf.getvalue())
def _send_dbopen(self, database: str) -> None:
"""Emit a SQ_DBOPEN PDU per JDBC's executeOpenDatabase.
Layout (from capture analysis):
``[short SQ_DBOPEN=36][short nameLen][bytes name][byte 0 if odd-len pad][short mode=0][short SQ_EOT]``
"""
writer, buf = make_pdu_writer()
writer.write_short(MessageType.SQ_DBOPEN)
name_bytes = database.encode("iso-8859-1")
writer.write_short(len(name_bytes))
writer.write_padded(name_bytes) # writes bytes + nul if odd
writer.write_short(0) # mode = 0 (default — read/write access)
writer.write_short(MessageType.SQ_EOT)
self._sock.write_all(buf.getvalue())
def _drain_to_eot(self) -> None:
"""Read response messages until SQ_EOT, dispatching on tag.
Raises ``OperationalError`` on SQ_ERR. Most response payloads we
don't need for session init are skipped after a best-effort length
decode. The SQ_PROTOCOLS reply has its own format; SQ_DONE has
warnings/rowcount/rowid/serial.
"""
while True:
tag = struct.unpack("!h", self._sock.read_exact(2))[0]
if tag == MessageType.SQ_EOT:
return
elif tag == MessageType.SQ_PROTOCOLS:
# ``[short payloadLen][bytes payload][byte 0 if odd-len pad]``
# Then the loop continues and consumes the next tag (usually SQ_EOT).
payload_len = struct.unpack("!h", self._sock.read_exact(2))[0]
if payload_len > 0:
self._sock.read_exact(payload_len)
if payload_len & 1:
self._sock.read_exact(1) # writePadded's even-alignment pad
elif tag == MessageType.SQ_DONE:
# [short warnings][int rows][int rowid][int serial]
self._sock.read_exact(2 + 4 + 4 + 4)
elif tag == 55: # SQ_COST — server appends cost info; ignore
# [int cost1][int cost2]
self._sock.read_exact(4 + 4)
elif tag == MessageType.SQ_ERR:
self._raise_sq_err()
else:
# Unknown tag during session init — fail loudly so we notice
raise OperationalError(
f"unexpected wire tag during session init: 0x{tag:04x} ({tag})"
)
def _raise_sq_err(self) -> None:
"""Decode a SQ_ERR payload and raise OperationalError.
Per ``IfxSqli.receiveError``:
``[short sqlcode][short isamcode][int statementOffset][...]``
"""
sqlcode = struct.unpack("!h", self._sock.read_exact(2))[0]
isamcode = struct.unpack("!h", self._sock.read_exact(2))[0]
offset = struct.unpack("!i", self._sock.read_exact(4))[0] # noqa: F841
# Drain any remaining error payload (varies by sqlcode) until SQ_EOT.
# Best-effort: read shorts and discard until we hit 0x000c.
try:
while True:
next_tag = struct.unpack("!h", self._sock.read_exact(2))[0]
if next_tag == MessageType.SQ_EOT:
break
except OperationalError:
pass
raise OperationalError(f"server returned SQ_ERR sqlcode={sqlcode} isamcode={isamcode}")
# -- login PDU assembly ------------------------------------------------
def _build_login_pdu(self, user: str, password: str | None) -> bytes:
def _build_login_pdu(
self, user: str, password: str | None, *, login_database: str | None = None
) -> bytes:
"""Assemble the full client→server login PDU.
Returns the SLheader (6 bytes) prepended to the PFheader payload.
@ -174,15 +326,15 @@ class Connection:
"""
# Build the PFheader (variable-size body).
pf_writer, pf_buf = make_pdu_writer()
self._write_pf_payload(pf_writer, user, password)
self._write_pf_payload(pf_writer, user, password, login_database=login_database)
pf_bytes = pf_buf.getvalue()
# Prepend the SLheader (6 bytes: total length, type, attr, opts).
sl_writer, sl_buf = make_pdu_writer()
sl_writer.write_short(len(pf_bytes) + 6) # total PDU size incl. header
sl_writer.write_byte(SLHeader.SLTYPE_CONREQ) # 1 = connection request
sl_writer.write_short(len(pf_bytes) + 6) # total PDU size incl. header
sl_writer.write_byte(SLHeader.SLTYPE_CONREQ) # 1 = connection request
sl_writer.write_byte(SLHeader.PF_PROT_SQLI_0600) # 60 = protocol version
sl_writer.write_short(0) # slOptions (0 in vanilla connect)
sl_writer.write_short(0) # slOptions (0 in vanilla connect)
return sl_buf.getvalue() + pf_bytes
@ -191,6 +343,8 @@ class Connection:
w: IfxStreamWriter,
user: str,
password: str | None,
*,
login_database: str | None = None,
) -> None:
"""Write the PFheader (binary login body), per PROTOCOL_NOTES §3b."""
# Association markers
@ -247,11 +401,13 @@ class Connection:
# Server name
w.write_string_with_nul(self._server)
# Database (optional)
if self._database is None:
# Database — always None in the login PDU per the JDBC behavior
# documented in __init__. The user-supplied database opens via
# SQ_DBOPEN in `_init_session`.
if login_database is None:
w.write_short(0)
else:
w.write_string_with_nul(self._database)
w.write_string_with_nul(login_database)
# 4 reserved/empty option slots (8 bytes total)
for _ in range(4):
@ -305,9 +461,7 @@ class Connection:
length_bytes = self._sock.read_exact(2)
total_length = struct.unpack("!h", length_bytes)[0]
if total_length < 6:
raise ProtocolError(
f"login response too short: {total_length} bytes"
)
raise ProtocolError(f"login response too short: {total_length} bytes")
# Read the rest of the SLheader + payload
rest = self._sock.read_exact(total_length - 2)
@ -346,9 +500,7 @@ class Connection:
# Then SQ_ASCBPARMS marker
marker = reader.read_short()
if marker != LoginMarker.SQ_ASCBPARMS:
raise OperationalError(
"server rejected the connection (no decodable error block)"
)
raise OperationalError("server rejected the connection (no decodable error block)")
# Skip 12 bytes of fixed-position metadata, then the version
# string, serial, applid, capabilities — we don't need any of
# that on the failure path, so we just bail out with a clear

View File

@ -1,17 +1,27 @@
"""DB-API 2.0 Cursor — SELECT execution and row iteration.
Phase 2 implements the simplest viable cursor: ``execute(sql)`` sends
``SQ_COMMAND`` (execute-immediate, no parameter binding) and the
response loop dispatches on tag (``SQ_DESCRIBE``, ``SQ_TUPLE``,
``SQ_DONE``, ``SQ_ERR``, ``SQ_EOT``). Parameter binding lands in
Phase 4 via ``SQ_PREPARE`` + ``SQ_BIND`` + ``SQ_EXECUTE``.
Phase 2 implements the full JDBC cursor lifecycle for parameterless SELECTs:
C PREPARE + NDESCRIBE + WANTDONE (one PDU)
C DESCRIBE (column metadata) + DONE + COST + EOT
C SQ_ID(CURNAME) + SQ_ID(NFETCH 4096) (one PDU)
C TUPLE* + DONE + COST + EOT (rows + completion)
C SQ_ID(NFETCH 4096) (drain to end)
C DONE + COST + EOT (no more rows)
C SQ_ID(CLOSE) (close cursor)
C EOT
C SQ_ID(RELEASE) (release statement)
C EOT
Parameter binding (SQ_BIND inserted between PREPARE and CURNAME) lands
in Phase 4.
"""
from __future__ import annotations
import itertools
import struct
from collections.abc import Iterator
from io import BytesIO
from typing import TYPE_CHECKING, Any
from ._messages import MessageType
@ -28,15 +38,25 @@ if TYPE_CHECKING:
from .connections import Connection
class Cursor:
"""PEP 249 Cursor over a SQLI session.
# Process-wide cursor name counter — appended to a "_ifxc" prefix to mimic
# JDBC's auto-generated names.
_cursor_counter = itertools.count(1)
One Cursor per Connection per active query is the simplest pattern
in Phase 2 (Phase 4's prepared-statement cache will share Cursors
across executes of the same SQL).
def _generate_cursor_name() -> str:
"""Generate a unique cursor name per the JDBC convention.
JDBC names are "_ifxc" + zero-padded counter, total 18 characters.
We replicate the format so the server treats us identically.
"""
n = next(_cursor_counter)
return f"_ifxc{n:013d}" # 5-char prefix + 13 digits = 18 chars
arraysize: int = 1 # PEP 249 default
class Cursor:
"""PEP 249 Cursor over a SQLI session."""
arraysize: int = 1
def __init__(self, connection: Connection):
self._conn = connection
@ -51,12 +71,10 @@ class Cursor:
@property
def description(self) -> list[tuple] | None:
"""Sequence of 7-tuples per PEP 249, one per result column. None pre-execute."""
return self._description
@property
def rowcount(self) -> int:
"""Affected/returned row count, -1 if not applicable or unknown."""
return self._rowcount
@property
@ -68,8 +86,8 @@ class Cursor:
def execute(self, operation: str, parameters: Any = None) -> None:
"""Execute a single SQL statement.
Phase 2 supports parameterless SQL only. Passing ``parameters``
raises ``NotSupportedError`` parameter binding lands in Phase 4.
Phase 2 supports parameterless SQL. ``parameters`` is reserved
for Phase 4 (SQ_BIND parameter binding).
"""
self._check_open()
if parameters is not None:
@ -77,16 +95,32 @@ class Cursor:
"parameter binding lands in Phase 4; pass SQL with literals for now"
)
# Reset previous-execute state
# Reset previous-execute state.
self._description = None
self._columns = []
self._rowcount = -1
self._rows = []
self._row_iter = None
pdu = self._build_command_pdu(operation)
self._conn._send_pdu(pdu)
self._read_response()
# Step 1: PREPARE — send SQL, receive column descriptors.
self._conn._send_pdu(self._build_prepare_pdu(operation))
self._read_describe_response()
# Step 2: open a cursor and fetch the first batch.
cursor_name = _generate_cursor_name()
self._conn._send_pdu(self._build_curname_nfetch_pdu(cursor_name))
self._read_fetch_response()
# Step 3: drain — fetch again to confirm no more rows.
# (JDBC always does this; the second fetch returns DONE only.)
self._conn._send_pdu(self._build_nfetch_pdu())
self._read_fetch_response() # appends rows if any; usually empty
# Step 4: close the cursor and release the prepared statement.
self._conn._send_pdu(self._build_close_pdu())
self._drain_to_eot()
self._conn._send_pdu(self._build_release_pdu())
self._drain_to_eot()
if self._description is not None:
self._row_iter = iter(self._rows)
@ -95,7 +129,6 @@ class Cursor:
raise NotSupportedError("executemany lands in Phase 4 (needs parameter binding)")
def fetchone(self) -> tuple | None:
"""Return the next row, or None when exhausted."""
self._check_open()
if self._row_iter is None:
return None
@ -147,95 +180,178 @@ class Cursor:
if self._conn.closed:
raise InterfaceError("connection is closed")
def _build_command_pdu(self, sql: str) -> bytes:
"""Assemble a SQ_COMMAND PDU per JDBC's sendCommand:
# -- PDU builders -----------------------------------------------------
[short SQ_COMMAND=1]
[short numValues=0]
[int sqlLen] 4-byte length (modern server)
[bytes sql]
[byte 0 if sqlLen+4 is odd] writeChar pad
[short SQ_NDESCRIBE=22]
[short SQ_EXECUTE=7]
[short SQ_RELEASE=11]
[short SQ_EOT=12]
def _build_prepare_pdu(self, sql: str) -> bytes:
"""SQ_PREPARE + SQ_NDESCRIBE + SQ_WANTDONE + SQ_EOT.
Per IfxSqli.sendPrepare. SQL uses 4-byte length prefix on modern
servers (isRemove64KLimitSupported), with even-byte alignment pad.
"""
writer, buf = make_pdu_writer()
writer.write_short(MessageType.SQ_COMMAND)
writer.write_short(0) # numValues — no bind parameters in Phase 2
writer.write_short(MessageType.SQ_PREPARE)
writer.write_short(0) # numQmarks — Phase 4 uses real values
sql_bytes = sql.encode("iso-8859-1")
writer.write_int(len(sql_bytes)) # 4-byte length prefix
writer.write_int(len(sql_bytes))
writer.write_bytes(sql_bytes)
# writeChar emits a 0x00 pad byte if total (4 + sqlLen) is odd
if (4 + len(sql_bytes)) & 1:
writer.write_byte(0)
writer.write_short(MessageType.SQ_NDESCRIBE) # 22
writer.write_short(MessageType.SQ_EXECUTE) # 7
writer.write_short(MessageType.SQ_RELEASE) # 11
writer.write_short(MessageType.SQ_EOT) # 12 — flush
writer.write_byte(0) # writeChar pad
writer.write_short(MessageType.SQ_NDESCRIBE)
writer.write_short(MessageType.SQ_WANTDONE)
writer.write_short(MessageType.SQ_EOT)
return buf.getvalue()
def _read_response(self) -> None:
"""Tag-driven response loop, mirrors JDBC's receiveMessage/dispatchMsg."""
# Wrap the connection's socket in a streaming reader that pulls
# from the wire on demand.
def _build_curname_nfetch_pdu(self, cursor_name: str) -> bytes:
"""SQ_ID(CURNAME) + SQ_ID(NFETCH 4096) chained.
From the JDBC capture (msg[21]):
[short SQ_ID=4][int 3][short nameLen][bytes name][short 6]
[short SQ_ID=4][int 9][int 4096][int 0]
[short SQ_EOT]
The trailing ``[short 6]`` after the cursor name is opaque
(cursor type / scrollability flag from JDBC's ``sendCursorName``);
we replay JDBC's value verbatim.
"""
writer, buf = make_pdu_writer()
# CURNAME
writer.write_short(MessageType.SQ_ID)
writer.write_int(MessageType.SQ_CURNAME) # action = 3
name_bytes = cursor_name.encode("ascii")
writer.write_short(len(name_bytes))
writer.write_bytes(name_bytes)
if len(name_bytes) & 1:
writer.write_byte(0)
writer.write_short(6) # cursor-type flag from JDBC
# NFETCH
writer.write_short(MessageType.SQ_ID)
writer.write_int(MessageType.SQ_NFETCH) # action = 9
writer.write_int(4096) # max bytes per fetch
writer.write_int(0) # reserved
writer.write_short(MessageType.SQ_EOT)
return buf.getvalue()
def _build_nfetch_pdu(self) -> bytes:
"""SQ_ID(NFETCH 4096) + SQ_EOT — used to drain remaining rows."""
writer, buf = make_pdu_writer()
writer.write_short(MessageType.SQ_ID)
writer.write_int(MessageType.SQ_NFETCH)
writer.write_int(4096)
writer.write_int(0)
writer.write_short(MessageType.SQ_EOT)
return buf.getvalue()
def _build_close_pdu(self) -> bytes:
"""SQ_ID(CLOSE) + SQ_EOT."""
writer, buf = make_pdu_writer()
writer.write_short(MessageType.SQ_ID)
writer.write_int(MessageType.SQ_CLOSE) # 10
writer.write_short(MessageType.SQ_EOT)
return buf.getvalue()
def _build_release_pdu(self) -> bytes:
"""SQ_ID(RELEASE) + SQ_EOT."""
writer, buf = make_pdu_writer()
writer.write_short(MessageType.SQ_ID)
writer.write_int(MessageType.SQ_RELEASE) # 11
writer.write_short(MessageType.SQ_EOT)
return buf.getvalue()
# -- response readers -------------------------------------------------
def _read_describe_response(self) -> None:
"""Read DESCRIBE + DONE + COST + EOT after a PREPARE."""
reader = _SocketReader(self._conn._sock)
while True:
tag = reader.read_short()
if tag == MessageType.SQ_EOT or tag == MessageType.SQ_EXIT:
break
if tag == MessageType.SQ_EOT:
return
elif tag == MessageType.SQ_DESCRIBE:
self._columns, _ = parse_describe(reader)
self._description = (
[c.to_description_tuple() for c in self._columns]
if self._columns else None
[c.to_description_tuple() for c in self._columns] if self._columns else None
)
elif tag == MessageType.SQ_DONE:
self._consume_done(reader)
elif tag == 55: # SQ_COST
reader.read_int()
reader.read_int()
elif tag == MessageType.SQ_ERR:
self._raise_sq_err(reader)
else:
raise DatabaseError(f"unexpected tag in DESCRIBE response: 0x{tag:04x}")
def _read_fetch_response(self) -> None:
"""Read TUPLE* + DONE + COST + EOT after an NFETCH."""
reader = _SocketReader(self._conn._sock)
while True:
tag = reader.read_short()
if tag == MessageType.SQ_EOT:
return
elif tag == MessageType.SQ_TUPLE:
row = parse_tuple_payload(reader, self._columns)
self._rows.append(row)
elif tag == MessageType.SQ_DONE:
self._read_done(reader)
self._consume_done(reader)
elif tag == 55: # SQ_COST
reader.read_int()
reader.read_int()
elif tag == MessageType.SQ_ERR:
self._raise_error(reader)
elif tag == 55: # SQ_COST — informational, ignore
# SQ_COST payload is two ints
reader.read_int()
reader.read_int()
self._raise_sq_err(reader)
else:
raise DatabaseError(f"unexpected wire tag in response: 0x{tag:04x}")
raise DatabaseError(f"unexpected tag in FETCH response: 0x{tag:04x}")
def _read_done(self, reader: IfxStreamReader) -> None:
"""SQ_DONE payload — see PROTOCOL_NOTES.md §6e (partial decode)."""
# Observed layout: [int 0][short rowcount][int sqlcode][int 0]
reader.read_int() # reserved
rc = reader.read_short()
sqlcode = reader.read_int() # noqa: F841 — Phase 5 surfaces this on errors
reader.read_int() # reserved
# rowcount is signed; -1 means unknown
self._rowcount = rc if rc >= 0 else -1
def _drain_to_eot(self) -> None:
"""Read response stream until SQ_EOT, allowing common tags in between."""
reader = _SocketReader(self._conn._sock)
while True:
tag = reader.read_short()
if tag == MessageType.SQ_EOT:
return
elif tag == MessageType.SQ_DONE:
self._consume_done(reader)
elif tag == 55:
reader.read_int()
reader.read_int()
elif tag == MessageType.SQ_ERR:
self._raise_sq_err(reader)
else:
raise DatabaseError(f"unexpected tag while draining: 0x{tag:04x}")
def _raise_error(self, reader: IfxStreamReader) -> None:
"""SQ_ERR — Phase 5 will decode SQLSTATE; for now raise generic."""
# Best effort: read whatever's there and surface as ProgrammingError
# which is the right class for "bad SQL" — most common error case.
def _consume_done(self, reader: IfxStreamReader) -> None:
"""SQ_DONE: [short warnings][int rowsAffected][int rowid][int serial]."""
reader.read_short() # warnings
rows = reader.read_int()
reader.read_int() # rowid
reader.read_int() # serial
if rows >= 0:
self._rowcount = rows
def _raise_sq_err(self, reader: IfxStreamReader) -> None:
"""Decode SQ_ERR per IfxSqli.receiveError and raise."""
sqlcode = reader.read_short()
isamcode = reader.read_short()
reader.read_int() # offset into statement
# Drain remaining error bytes until SQ_EOT.
try:
data = reader.read_exact(min(256, 4096))
while True:
t = reader.read_short()
if t == MessageType.SQ_EOT:
break
except Exception:
data = b""
raise ProgrammingError(
f"server returned SQ_ERR (full decode lands in Phase 5); "
f"first bytes: {data[:32].hex(' ')}"
)
pass
raise ProgrammingError(f"server returned SQ_ERR sqlcode={sqlcode} isamcode={isamcode}")
class _SocketReader(IfxStreamReader):
"""IfxStreamReader backed by an IfxSocket — pulls more bytes from the wire as needed."""
"""``IfxStreamReader`` backed by an ``IfxSocket`` — pulls bytes from the wire on demand."""
def __init__(self, sock):
self._sock = sock
# Initialize parent with a dummy BytesIO — we override read methods.
from io import BytesIO
super().__init__(BytesIO(b""))
def read_exact(self, n: int) -> bytes:

View File

@ -72,10 +72,14 @@ def ifx_connection(conn_params: ConnParams) -> Iterator[object]:
import informix_db
conn = informix_db.connect(
host=conn_params.host, port=conn_params.port,
user=conn_params.user, password=conn_params.password,
database=conn_params.database, server=conn_params.server,
connect_timeout=10.0, read_timeout=10.0,
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password=conn_params.password,
database=conn_params.database,
server=conn_params.server,
connect_timeout=10.0,
read_timeout=10.0,
)
try:
yield conn

View File

@ -72,9 +72,12 @@ def python_login_pdu(monkeypatch: pytest.MonkeyPatch) -> bytes:
with pytest.raises(informix_db.OperationalError, match="stub"):
informix_db.connect(
host="dont.care", port=9088,
user="informix", password="in4mix",
database=None, server="informix",
host="dont.care",
port=9088,
user="informix",
password="in4mix",
database=None,
server="informix",
)
return bytes(captured)
@ -97,9 +100,7 @@ def test_slheader_protocol_version_matches(
assert python_login_pdu[3] == jdbc_reference_pdu[3] == 0x3C
def test_slheader_type_byte_matches(
python_login_pdu: bytes, jdbc_reference_pdu: bytes
) -> None:
def test_slheader_type_byte_matches(python_login_pdu: bytes, jdbc_reference_pdu: bytes) -> None:
"""The SLheader's slType byte (offset 2) must be 1 (SLTYPE_CONREQ)."""
assert python_login_pdu[2] == jdbc_reference_pdu[2] == 0x01
@ -115,9 +116,7 @@ def test_capability_ints_match_reference(
assert python_login_pdu[65:77] == jdbc_reference_pdu[65:77]
def test_structural_prefix_matches(
python_login_pdu: bytes, jdbc_reference_pdu: bytes
) -> None:
def test_structural_prefix_matches(python_login_pdu: bytes, jdbc_reference_pdu: bytes) -> None:
"""Everything from byte 2 to ``STRUCTURAL_PREFIX_END`` must match exactly.
Skips:
@ -138,9 +137,9 @@ def test_structural_prefix_matches(
f"structural-prefix mismatch at offset {off}: "
f"Python={a:#04x} JDBC={b:#04x}\n"
f" Python[{off - 4}..{off + 4}]: "
f"{python_login_pdu[off - 4:off + 5].hex(' ')}\n"
f"{python_login_pdu[off - 4 : off + 5].hex(' ')}\n"
f" JDBC [{off - 4}..{off + 4}]: "
f"{jdbc_reference_pdu[off - 4:off + 5].hex(' ')}"
f"{jdbc_reference_pdu[off - 4 : off + 5].hex(' ')}"
)
assert py_prefix == ja_prefix

122
tests/test_select.py Normal file
View File

@ -0,0 +1,122 @@
"""Phase 2 integration tests — SELECT execution end-to-end.
Marked ``integration`` so the default ``pytest`` invocation skips them.
Run with ``pytest -m integration`` after ``docker compose up``.
"""
from __future__ import annotations
import pytest
import informix_db
from tests.conftest import ConnParams
pytestmark = pytest.mark.integration
def _connect(conn_params: ConnParams) -> informix_db.Connection:
return informix_db.connect(
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password=conn_params.password,
database=conn_params.database,
server=conn_params.server,
connect_timeout=10.0,
read_timeout=10.0,
)
def test_select_1_returns_one_tuple(conn_params: ConnParams) -> None:
"""The canonical Phase 2 milestone: ``SELECT 1`` → ``(1,)``."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("SELECT 1 FROM systables WHERE tabid = 1")
assert cur.fetchone() == (1,)
assert cur.fetchone() is None # no more rows
def test_select_1_description_shape(conn_params: ConnParams) -> None:
"""``cursor.description`` is a 7-tuple per PEP 249."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("SELECT 1 FROM systables WHERE tabid = 1")
assert cur.description is not None
assert len(cur.description) == 1
col = cur.description[0]
assert len(col) == 7
# (name, type_code, display_size, internal_size, precision, scale, null_ok)
name, type_code, display_size, internal_size, precision, scale, null_ok = col
assert name == "(constant)"
assert type_code == 2 # IfxType.INT
assert display_size == internal_size == 4
def test_select_multi_row_int(conn_params: ConnParams) -> None:
"""Multi-row INT SELECT — fetchall returns a list of tuples."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("SELECT FIRST 5 tabid FROM systables ORDER BY tabid")
rows = cur.fetchall()
assert len(rows) == 5
assert rows == [(1,), (2,), (3,), (4,), (5,)]
assert cur.rowcount == 5
def test_select_multi_column_mixed_types(conn_params: ConnParams) -> None:
"""Multi-column with mixed types (INT + FLOAT)."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("SELECT FIRST 3 tabid, nrows FROM systables ORDER BY tabid")
rows = cur.fetchall()
assert len(rows) == 3
for tabid, nrows in rows:
assert isinstance(tabid, int)
assert isinstance(nrows, float)
names = [c[0] for c in cur.description]
assert names == ["tabid", "nrows"]
def test_iterator_protocol(conn_params: ConnParams) -> None:
"""Cursor supports the iterator protocol — ``for row in cursor``."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("SELECT FIRST 3 tabid FROM systables ORDER BY tabid")
rows = list(cur)
assert rows == [(1,), (2,), (3,)]
def test_fetchmany(conn_params: ConnParams) -> None:
"""``fetchmany(n)`` returns up to n rows."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("SELECT FIRST 5 tabid FROM systables ORDER BY tabid")
first_two = cur.fetchmany(2)
assert first_two == [(1,), (2,)]
rest = cur.fetchall()
assert rest == [(3,), (4,), (5,)]
def test_two_executes_on_same_cursor(conn_params: ConnParams) -> None:
"""Re-executing on the same cursor resets state cleanly."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("SELECT 1 FROM systables WHERE tabid = 1")
assert cur.fetchone() == (1,)
cur.execute("SELECT 2 FROM systables WHERE tabid = 1")
assert cur.fetchone() == (2,)
def test_two_cursors_on_same_connection(conn_params: ConnParams) -> None:
"""Two cursors on one connection — used sequentially (Phase 4 may parallel-ize)."""
with _connect(conn_params) as conn:
cur1 = conn.cursor()
cur1.execute("SELECT 1 FROM systables WHERE tabid = 1")
assert cur1.fetchone() == (1,)
cur1.close()
cur2 = conn.cursor()
cur2.execute("SELECT FIRST 2 tabid FROM systables ORDER BY tabid")
assert cur2.fetchall() == [(1,), (2,)]
cur2.close()

View File

@ -18,10 +18,14 @@ pytestmark = pytest.mark.integration
def test_connect_and_close(conn_params: ConnParams) -> None:
"""The happy path: connect with valid creds, then close cleanly."""
conn = informix_db.connect(
host=conn_params.host, port=conn_params.port,
user=conn_params.user, password=conn_params.password,
database=conn_params.database, server=conn_params.server,
connect_timeout=10.0, read_timeout=10.0,
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password=conn_params.password,
database=conn_params.database,
server=conn_params.server,
connect_timeout=10.0,
read_timeout=10.0,
)
assert conn.closed is False
conn.close()
@ -31,9 +35,12 @@ def test_connect_and_close(conn_params: ConnParams) -> None:
def test_close_is_idempotent(conn_params: ConnParams) -> None:
"""``close()`` must be safely callable multiple times."""
conn = informix_db.connect(
host=conn_params.host, port=conn_params.port,
user=conn_params.user, password=conn_params.password,
database=conn_params.database, server=conn_params.server,
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password=conn_params.password,
database=conn_params.database,
server=conn_params.server,
connect_timeout=10.0,
)
conn.close()
@ -44,9 +51,12 @@ def test_close_is_idempotent(conn_params: ConnParams) -> None:
def test_context_manager(conn_params: ConnParams) -> None:
"""``with`` block closes on exit."""
with informix_db.connect(
host=conn_params.host, port=conn_params.port,
user=conn_params.user, password=conn_params.password,
database=conn_params.database, server=conn_params.server,
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password=conn_params.password,
database=conn_params.database,
server=conn_params.server,
connect_timeout=10.0,
) as conn:
assert conn.closed is False
@ -57,10 +67,14 @@ def test_bad_password_raises_operational_error(conn_params: ConnParams) -> None:
"""Auth failure must be ``OperationalError``, not a generic socket error."""
with pytest.raises(informix_db.OperationalError):
informix_db.connect(
host=conn_params.host, port=conn_params.port,
user=conn_params.user, password="definitely-the-wrong-password",
database=conn_params.database, server=conn_params.server,
connect_timeout=5.0, read_timeout=5.0,
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password="definitely-the-wrong-password",
database=conn_params.database,
server=conn_params.server,
connect_timeout=5.0,
read_timeout=5.0,
)
@ -69,8 +83,12 @@ def test_bad_host_raises_operational_error(conn_params: ConnParams) -> None:
# Pick an unused port on loopback.
with pytest.raises(informix_db.OperationalError, match="cannot connect"):
informix_db.connect(
host="127.0.0.1", port=1, # IANA-reserved, nothing listens
user="x", password="x", database="x", server="x",
host="127.0.0.1",
port=1, # IANA-reserved, nothing listens
user="x",
password="x",
database="x",
server="x",
connect_timeout=2.0,
)
@ -78,9 +96,12 @@ def test_bad_host_raises_operational_error(conn_params: ConnParams) -> None:
def test_cursor_returns_cursor_object(conn_params: ConnParams) -> None:
"""Phase 2: ``cursor()`` returns a Cursor; SELECT execution is partial work-in-progress."""
with informix_db.connect(
host=conn_params.host, port=conn_params.port,
user=conn_params.user, password=conn_params.password,
database=conn_params.database, server=conn_params.server,
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password=conn_params.password,
database=conn_params.database,
server=conn_params.server,
connect_timeout=10.0,
) as conn:
cur = conn.cursor()
@ -95,9 +116,12 @@ def test_cursor_returns_cursor_object(conn_params: ConnParams) -> None:
def test_cursor_with_parameters_raises(conn_params: ConnParams) -> None:
"""Parameter binding lands in Phase 4; passing parameters must raise NotSupportedError."""
with informix_db.connect(
host=conn_params.host, port=conn_params.port,
user=conn_params.user, password=conn_params.password,
database=conn_params.database, server=conn_params.server,
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password=conn_params.password,
database=conn_params.database,
server=conn_params.server,
connect_timeout=10.0,
) as conn:
cur = conn.cursor()