warehack.ing/informix-db
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
informix-db/docs/CAPTURES/18-py-null-rows.socat.log
Ryan Malloy d508a489fd Phase 4.x: parameterized SELECT, NULL row decoding, executemany() 
Three Phase 4 follow-ups in one push, all with empirical wire analysis:

1. PARAMETERIZED SELECT
   cur.execute('SELECT tabname FROM systables WHERE tabid = ?', (1,))
   → ('systables',)
   Wire flow: PREPARE → DESCRIBE → SQ_BIND-only (no EXECUTE) →
   CURNAME+NFETCH → TUPLE+DONE → drain → CLOSE+RELEASE.
   The cursor open is what executes the prepared query; SQ_BIND just
   binds values into scope. No need for the IDESCRIBE handshake JDBC
   does for type discovery — server accepts our typed bind directly.

2. NULL ROW DECODING — per-type sentinel detection
   Each IDS type has its own NULL sentinel in tuple data:
     INT     → 0x80000000 (INT_MIN)
     BIGINT  → 0x8000000000000000 (LONG_MIN)
     SMALLINT→ 0x8000 (SHORT_MIN)
     REAL    → all 0xFF (NaN bit pattern)
     FLOAT   → all 0xFF
     DATE    → 0x80000000 (same as INT)
     VARCHAR → [byte 1][byte 0]  (length=1, single nul) — distinguishable
                from empty '' which is [byte 0] (length=0)
   Verified by wire capture against the dev container — see
   docs/CAPTURES/19-py-null-vs-onechar.socat.log and
   docs/CAPTURES/20-py-int-null.socat.log.

   The VARCHAR null marker is the trickiest because it LOOKS like a
   1-byte string of nul, but VARCHAR can't contain embedded nuls
   anyway, so the byte-0 within length-1 is unambiguous.

3. executemany(sql, seq_of_params) — PEP 249 batched DML
   PREPARE once, loop SQ_BIND+SQ_EXECUTE per param set, RELEASE once.
   Performance: only ~1.06x faster than execute() loop for 200 INSERTs
   (dominated by per-row round trips). Phase 4.x optimization opportunity:
   chain BIND+EXECUTE in one PDU without intermediate flush+read for
   true bulk performance (would likely give 5-10x). Documented in
   DECISION_LOG.md as a follow-up.

Module changes:
  src/informix_db/converters.py:
    + Per-type NULL sentinel constants and detection in each decoder
    + Decoders now return None for sentinel values
  src/informix_db/cursors.py:
    + _execute_select_with_params() — SQ_BIND alone, then cursor open
    + _build_bind_only_pdu() — SQ_BIND without trailing SQ_EXECUTE
    + executemany() — loop BIND+EXECUTE, accumulate rowcount
    + execute() now dispatches to _execute_select_with_params for
      parameterized SELECT (was: NotSupportedError)

Tests: 40 unit + 47 integration (was 32; added 15 new) = 87 total,
all green, ruff clean. New test files / cases:
  tests/test_nulls.py (7) — NULL decoding for INT, BIGINT, FLOAT,
    REAL, VARCHAR, empty-vs-null, mixed columns
  tests/test_params.py — added 4 parameterized SELECT tests, 5
    executemany tests
  tests/test_smoke.py — updated cursor-with-params test (was Phase 1
    "raises", now Phase 4 "works")

Discovered captures kept for next-session debugging:
  docs/CAPTURES/18-py-null-rows.socat.log
  docs/CAPTURES/19-py-null-vs-onechar.socat.log
  docs/CAPTURES/20-py-int-null.socat.log
2026-05-04 11:11:50 -06:00

87 lines
8.6 KiB
Plaintext
Raw Blame History

2026/05/04 11:06:27 socat[816537] N listening on AF=2 0.0.0.0:9090
2026/05/04 11:06:28 socat[816537] N accepting connection from AF=2 127.0.0.1:53772 on AF=2 127.0.0.1:9090
2026/05/04 11:06:28 socat[816537] N opening connection to 127.0.0.1:9088
2026/05/04 11:06:28 socat[816537] N opening connection to AF=2 127.0.0.1:9088
2026/05/04 11:06:28 socat[816537] N successfully connected from local address AF=2 127.0.0.1:45370
2026/05/04 11:06:28 socat[816537] N successfully connected to 127.0.0.1:9088
2026/05/04 11:06:28 socat[816537] N starting data transfer loop with FDs [6,6] and [5,5]
> 2026/05/04 11:06:28.233803 length=384 from=0 to=383
01 80 01 3c 00 00 00 64 00 65 00 00 00 3d 00 06 49 45 45 45 4d 00 00 6c 73 71 6c 65 78 65 63 00 00 00 00 00 00 06 39 2e 32 38 30 00 00 0c 52 44 53 23 52 30 30 30 30 30 30 00 00 05 73 71 6c 69 00 00 00 01 3c 00 00 00 00 00 00 00 00 00 01 00 09 69 6e 66 6f 72 6d 69 78 00 00 07 69 6e 34 6d 69 78 00 6f 6c 00 00 00 00 00 00 00 00 00 3d 74 6c 69 74 63 70 00 00 00 00 00 01 00 68 00 0b 00 00 00 03 00 09 69 6e 66 6f 72 6d 69 78 00 00 00 00 00 00 00 00 00 00 00 00 6a 00 06 00 07 44 42 50 41 54 48 00 00 02 2e 00 00 0e 43 4c 49 45 4e 54 5f 4c 4f 43 41 4c 45 00 00 0d 65 6e 5f 55 53 2e 38 38 35 39 2d 31 00 00 11 43 4c 4e 54 5f 50 41 4d 5f 43 41 50 41 42 4c 45 00 00 02 31 00 00 07 44 42 44 41 54 45 00 00 06 59 34 4d 44 2d 00 00 0c 49 46 58 5f 55 50 44 44 45 53 43 00 00 02 31 00 00 09 4e 4f 44 45 46 44 41 43 00 00 03 6e 6f 00 00 6b 00 00 00 00 00 0c 75 ae 00 00 00 00 00 0b 72 70 6d 2d 62 75 6c 6c 65 74 00 00 00 00 29 2f 68 6f 6d 65 2f 72 70 6d 2f 63 6c 61 75 64 65 2f 69 6e 66 6f 72 6d 69 78 2f 70 79 74 68 6f 6e 2d 6c 69 62 72 61 72 79 00 00 74 00 20 00 00 00 00 00 00 00 00 00 16 69 6e 66 6f 72 6d 69 78 2d 64 62 40 70 69 64 38 31 36 35 35 38 00 00 7f
< 2026/05/04 11:06:28.245386 length=276 from=0 to=275
01 14 02 3c 10 00 00 64 00 65 00 00 00 3d 00 06 49 45 45 45 49 00 00 6c 73 72 76 69 6e 66 78 00 00 00 00 00 00 2f 49 42 4d 20 49 6e 66 6f 72 6d 69 78 20 44 79 6e 61 6d 69 63 20 53 65 72 76 65 72 20 56 65 72 73 69 6f 6e 20 31 35 2e 30 2e 31 2e 30 2e 33 00 00 07 73 65 72 69 61 6c 00 00 09 69 6e 66 6f 72 6d 69 78 00 00 00 01 3c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 6f 6e 00 00 00 00 00 00 00 00 00 3d 73 6f 63 74 63 70 00 00 00 00 00 00 00 66 00 00 00 00 00 00 00 00 00 00 00 14 00 00 00 6b 00 00 00 00 00 00 03 1a 00 00 00 00 00 0d 32 33 32 37 63 34 33 35 34 65 61 38 00 00 00 00 0f 2f 68 6f 6d 65 2f 69 6e 66 6f 72 6d 69 78 00 00 6e 00 04 00 00 00 00 00 74 00 33 00 00 00 c8 00 00 00 c8 00 29 2f 6f 70 74 2f 69 62 6d 2f 69 6e 66 6f 72 6d 69 78 2f 76 31 35 2e 30 2e 31 2e 30 2e 33 2f 62 69 6e 2f 6f 6e 69 6e 69 74 00 00 7f
> 2026/05/04 11:06:28.245659 length=14 from=384 to=397
00 7e 00 08 ff fc 7f fc 3c 8c aa 97 00 0c
< 2026/05/04 11:06:28.245700 length=16 from=276 to=291
00 7e 00 09 bd be 9f fe 7f b7 ff ef ff 00 00 0c
> 2026/05/04 11:06:28.245728 length=48 from=398 to=445
00 51 00 06 00 26 00 0c 00 04 00 06 44 42 54 45 4d 50 00 04 2f 74 6d 70 00 0b 53 55 42 51 43 41 43 48 45 53 5a 00 00 02 31 30 00 00 00 00 00 0c
< 2026/05/04 11:06:28.245788 length=2 from=292 to=293
00 0c
> 2026/05/04 11:06:28.245804 length=18 from=446 to=463
00 24 00 09 73 79 73 6d 61 73 74 65 72 00 00 00 00 0c
< 2026/05/04 11:06:28.245963 length=28 from=294 to=321
00 0f 00 15 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:06:28.246010 length=76 from=464 to=539
00 02 00 00 00 00 00 3d 43 52 45 41 54 45 20 54 45 4d 50 20 54 41 42 4c 45 20 74 20 28 69 64 20 49 4e 54 45 47 45 52 2c 20 6e 61 6d 65 20 56 41 52 43 48 41 52 28 35 30 29 2c 20 76 61 6c 20 46 4c 4f 41 54 29 00 00 16 00 31 00 0c
< 2026/05/04 11:06:28.246124 length=46 from=322 to=367
00 08 00 2d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:06:28.246180 length=8 from=540 to=547
00 04 00 00 00 07 00 0c
< 2026/05/04 11:06:28.251154 length=28 from=368 to=395
00 0f 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:06:28.251235 length=8 from=548 to=555
00 04 00 00 00 0b 00 0c
< 2026/05/04 11:06:28.251280 length=2 from=396 to=397
00 0c
> 2026/05/04 11:06:28.251306 length=44 from=556 to=599
00 02 00 03 00 00 00 1e 49 4e 53 45 52 54 20 49 4e 54 4f 20 74 20 56 41 4c 55 45 53 20 28 3f 2c 20 3f 2c 20 3f 29 00 16 00 31 00 0c
< 2026/05/04 11:06:28.251407 length=168 from=398 to=565
00 08 00 06 00 00 00 00 00 00 00 3f 00 03 00 00 00 0c 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 00 00 00 03 00 00 00 04 00 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 32 00 00 00 08 00 00 00 37 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08 69 64 00 6e 61 6d 65 00 76 61 6c 00 00 5e 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:06:28.251591 length=50 from=600 to=649
00 04 00 00 00 05 00 03 00 02 00 00 0a 00 00 00 00 01 00 00 00 00 00 00 00 05 68 65 6c 6c 6f 00 00 03 00 00 00 00 40 09 1e b8 51 eb 85 1f 00 07 00 0c
< 2026/05/04 11:06:28.253369 length=48 from=566 to=613
00 5e 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 00 00 00 00 01 00 00 01 01 00 00 00 00 00 37 00 00 00 01 00 00 00 02 00 0c
> 2026/05/04 11:06:28.253424 length=8 from=650 to=657
00 04 00 00 00 0b 00 0c
< 2026/05/04 11:06:28.253462 length=2 from=614 to=615
00 0c
> 2026/05/04 11:06:28.253485 length=44 from=658 to=701
00 02 00 03 00 00 00 1e 49 4e 53 45 52 54 20 49 4e 54 4f 20 74 20 56 41 4c 55 45 53 20 28 3f 2c 20 3f 2c 20 3f 29 00 16 00 31 00 0c
< 2026/05/04 11:06:28.253564 length=168 from=616 to=783
00 08 00 06 00 00 00 00 00 00 00 3f 00 03 00 00 00 0c 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 00 00 00 03 00 00 00 04 00 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 32 00 00 00 08 00 00 00 37 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08 69 64 00 6e 61 6d 65 00 76 61 6c 00 00 5e 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:06:28.253710 length=34 from=702 to=735
00 04 00 00 00 05 00 03 00 02 00 00 0a 00 00 00 00 02 00 00 ff ff 00 00 00 00 ff ff 00 00 00 07 00 0c
< 2026/05/04 11:06:28.255614 length=48 from=784 to=831
00 5e 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 00 00 00 00 01 00 00 01 02 00 00 00 00 00 37 00 00 00 01 00 00 00 02 00 0c
> 2026/05/04 11:06:28.255676 length=8 from=736 to=743
00 04 00 00 00 0b 00 0c
< 2026/05/04 11:06:28.255715 length=2 from=832 to=833
00 0c
> 2026/05/04 11:06:28.255734 length=54 from=744 to=797
00 02 00 00 00 00 00 27 53 45 4c 45 43 54 20 69 64 2c 20 6e 61 6d 65 2c 20 76 61 6c 20 46 52 4f 4d 20 74 20 4f 52 44 45 52 20 42 59 20 69 64 00 00 16 00 31 00 0c
< 2026/05/04 11:06:28.255890 length=148 from=834 to=981
00 08 00 02 00 00 00 00 00 00 00 3f 00 03 00 00 00 0c 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 00 00 00 03 00 00 00 04 00 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 32 00 00 00 08 00 00 00 37 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08 69 64 00 6e 61 6d 65 00 76 61 6c 00 00 0f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 02 00 00 00 02 00 0c
> 2026/05/04 11:06:28.256035 length=42 from=798 to=839
00 04 00 00 00 03 00 12 5f 69 66 78 63 30 30 30 30 30 30 30 30 30 30 30 30 31 00 06 00 04 00 00 00 09 00 00 10 00 00 00 00 0c
< 2026/05/04 11:06:28.256137 length=76 from=982 to=1057
00 0e 00 00 00 00 00 12 00 00 00 01 05 68 65 6c 6c 6f 40 09 1e b8 51 eb 85 1f 00 0e 00 00 00 00 00 0e 00 00 00 02 01 00 ff ff ff ff ff ff ff ff 00 0f 00 00 00 00 00 02 00 00 01 02 00 00 00 00 00 37 00 00 00 02 00 00 00 02 00 0c
> 2026/05/04 11:06:28.256224 length=14 from=840 to=853
00 04 00 00 00 09 00 00 10 00 00 00 00 0c
< 2026/05/04 11:06:28.256252 length=28 from=1058 to=1085
00 0f 00 00 00 00 00 02 00 00 01 02 00 00 00 00 00 37 00 00 00 02 00 00 00 02 00 0c
> 2026/05/04 11:06:28.256286 length=8 from=854 to=861
00 04 00 00 00 0a 00 0c
< 2026/05/04 11:06:28.256312 length=2 from=1086 to=1087
00 0c
> 2026/05/04 11:06:28.256325 length=8 from=862 to=869
00 04 00 00 00 0b 00 0c
< 2026/05/04 11:06:28.256351 length=2 from=1088 to=1089
00 0c
> 2026/05/04 11:06:28.256386 length=2 from=870 to=871
00 38
< 2026/05/04 11:06:28.259522 length=2 from=1090 to=1091
00 38
2026/05/04 11:06:28 socat[816537] N socket 1 (fd 6) is at EOF
2026/05/04 11:06:28 socat[816537] N socket 2 (fd 5) is at EOF
2026/05/04 11:06:28 socat[816537] N exiting with status 0
Reference in New Issue View Git Blame Copy Permalink