warehack.ing/informix-db
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
informix-db/docs/CAPTURES/19-py-null-vs-onechar.socat.log
Ryan Malloy d508a489fd Phase 4.x: parameterized SELECT, NULL row decoding, executemany() 
Three Phase 4 follow-ups in one push, all with empirical wire analysis:

1. PARAMETERIZED SELECT
   cur.execute('SELECT tabname FROM systables WHERE tabid = ?', (1,))
   → ('systables',)
   Wire flow: PREPARE → DESCRIBE → SQ_BIND-only (no EXECUTE) →
   CURNAME+NFETCH → TUPLE+DONE → drain → CLOSE+RELEASE.
   The cursor open is what executes the prepared query; SQ_BIND just
   binds values into scope. No need for the IDESCRIBE handshake JDBC
   does for type discovery — server accepts our typed bind directly.

2. NULL ROW DECODING — per-type sentinel detection
   Each IDS type has its own NULL sentinel in tuple data:
     INT     → 0x80000000 (INT_MIN)
     BIGINT  → 0x8000000000000000 (LONG_MIN)
     SMALLINT→ 0x8000 (SHORT_MIN)
     REAL    → all 0xFF (NaN bit pattern)
     FLOAT   → all 0xFF
     DATE    → 0x80000000 (same as INT)
     VARCHAR → [byte 1][byte 0]  (length=1, single nul) — distinguishable
                from empty '' which is [byte 0] (length=0)
   Verified by wire capture against the dev container — see
   docs/CAPTURES/19-py-null-vs-onechar.socat.log and
   docs/CAPTURES/20-py-int-null.socat.log.

   The VARCHAR null marker is the trickiest because it LOOKS like a
   1-byte string of nul, but VARCHAR can't contain embedded nuls
   anyway, so the byte-0 within length-1 is unambiguous.

3. executemany(sql, seq_of_params) — PEP 249 batched DML
   PREPARE once, loop SQ_BIND+SQ_EXECUTE per param set, RELEASE once.
   Performance: only ~1.06x faster than execute() loop for 200 INSERTs
   (dominated by per-row round trips). Phase 4.x optimization opportunity:
   chain BIND+EXECUTE in one PDU without intermediate flush+read for
   true bulk performance (would likely give 5-10x). Documented in
   DECISION_LOG.md as a follow-up.

Module changes:
  src/informix_db/converters.py:
    + Per-type NULL sentinel constants and detection in each decoder
    + Decoders now return None for sentinel values
  src/informix_db/cursors.py:
    + _execute_select_with_params() — SQ_BIND alone, then cursor open
    + _build_bind_only_pdu() — SQ_BIND without trailing SQ_EXECUTE
    + executemany() — loop BIND+EXECUTE, accumulate rowcount
    + execute() now dispatches to _execute_select_with_params for
      parameterized SELECT (was: NotSupportedError)

Tests: 40 unit + 47 integration (was 32; added 15 new) = 87 total,
all green, ruff clean. New test files / cases:
  tests/test_nulls.py (7) — NULL decoding for INT, BIGINT, FLOAT,
    REAL, VARCHAR, empty-vs-null, mixed columns
  tests/test_params.py — added 4 parameterized SELECT tests, 5
    executemany tests
  tests/test_smoke.py — updated cursor-with-params test (was Phase 1
    "raises", now Phase 4 "works")

Discovered captures kept for next-session debugging:
  docs/CAPTURES/18-py-null-rows.socat.log
  docs/CAPTURES/19-py-null-vs-onechar.socat.log
  docs/CAPTURES/20-py-int-null.socat.log
2026-05-04 11:11:50 -06:00

99 lines
8.9 KiB
Plaintext
Raw Blame History

2026/05/04 11:07:56 socat[837404] N listening on AF=2 0.0.0.0:9090
2026/05/04 11:07:57 socat[837404] N accepting connection from AF=2 127.0.0.1:52758 on AF=2 127.0.0.1:9090
2026/05/04 11:07:57 socat[837404] N opening connection to 127.0.0.1:9088
2026/05/04 11:07:57 socat[837404] N opening connection to AF=2 127.0.0.1:9088
2026/05/04 11:07:57 socat[837404] N successfully connected from local address AF=2 127.0.0.1:60228
2026/05/04 11:07:57 socat[837404] N successfully connected to 127.0.0.1:9088
2026/05/04 11:07:57 socat[837404] N starting data transfer loop with FDs [6,6] and [5,5]
> 2026/05/04 11:07:57.318852 length=384 from=0 to=383
01 80 01 3c 00 00 00 64 00 65 00 00 00 3d 00 06 49 45 45 45 4d 00 00 6c 73 71 6c 65 78 65 63 00 00 00 00 00 00 06 39 2e 32 38 30 00 00 0c 52 44 53 23 52 30 30 30 30 30 30 00 00 05 73 71 6c 69 00 00 00 01 3c 00 00 00 00 00 00 00 00 00 01 00 09 69 6e 66 6f 72 6d 69 78 00 00 07 69 6e 34 6d 69 78 00 6f 6c 00 00 00 00 00 00 00 00 00 3d 74 6c 69 74 63 70 00 00 00 00 00 01 00 68 00 0b 00 00 00 03 00 09 69 6e 66 6f 72 6d 69 78 00 00 00 00 00 00 00 00 00 00 00 00 6a 00 06 00 07 44 42 50 41 54 48 00 00 02 2e 00 00 0e 43 4c 49 45 4e 54 5f 4c 4f 43 41 4c 45 00 00 0d 65 6e 5f 55 53 2e 38 38 35 39 2d 31 00 00 11 43 4c 4e 54 5f 50 41 4d 5f 43 41 50 41 42 4c 45 00 00 02 31 00 00 07 44 42 44 41 54 45 00 00 06 59 34 4d 44 2d 00 00 0c 49 46 58 5f 55 50 44 44 45 53 43 00 00 02 31 00 00 09 4e 4f 44 45 46 44 41 43 00 00 03 6e 6f 00 00 6b 00 00 00 00 00 0c c7 30 00 00 00 00 00 0b 72 70 6d 2d 62 75 6c 6c 65 74 00 00 00 00 29 2f 68 6f 6d 65 2f 72 70 6d 2f 63 6c 61 75 64 65 2f 69 6e 66 6f 72 6d 69 78 2f 70 79 74 68 6f 6e 2d 6c 69 62 72 61 72 79 00 00 74 00 20 00 00 00 00 00 00 00 00 00 16 69 6e 66 6f 72 6d 69 78 2d 64 62 40 70 69 64 38 33 37 34 32 34 00 00 7f
< 2026/05/04 11:07:57.322619 length=276 from=0 to=275
01 14 02 3c 10 00 00 64 00 65 00 00 00 3d 00 06 49 45 45 45 49 00 00 6c 73 72 76 69 6e 66 78 00 00 00 00 00 00 2f 49 42 4d 20 49 6e 66 6f 72 6d 69 78 20 44 79 6e 61 6d 69 63 20 53 65 72 76 65 72 20 56 65 72 73 69 6f 6e 20 31 35 2e 30 2e 31 2e 30 2e 33 00 00 07 73 65 72 69 61 6c 00 00 09 69 6e 66 6f 72 6d 69 78 00 00 00 01 3c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 6f 6e 00 00 00 00 00 00 00 00 00 3d 73 6f 63 74 63 70 00 00 00 00 00 00 00 66 00 00 00 00 00 00 00 00 00 00 00 14 00 00 00 6b 00 00 00 00 00 00 03 1a 00 00 00 00 00 0d 32 33 32 37 63 34 33 35 34 65 61 38 00 00 00 00 0f 2f 68 6f 6d 65 2f 69 6e 66 6f 72 6d 69 78 00 00 6e 00 04 00 00 00 00 00 74 00 33 00 00 00 c8 00 00 00 c8 00 29 2f 6f 70 74 2f 69 62 6d 2f 69 6e 66 6f 72 6d 69 78 2f 76 31 35 2e 30 2e 31 2e 30 2e 33 2f 62 69 6e 2f 6f 6e 69 6e 69 74 00 00 7f
> 2026/05/04 11:07:57.322854 length=14 from=384 to=397
00 7e 00 08 ff fc 7f fc 3c 8c aa 97 00 0c
< 2026/05/04 11:07:57.329158 length=16 from=276 to=291
00 7e 00 09 bd be 9f fe 7f b7 ff ef ff 00 00 0c
> 2026/05/04 11:07:57.329205 length=48 from=398 to=445
00 51 00 06 00 26 00 0c 00 04 00 06 44 42 54 45 4d 50 00 04 2f 74 6d 70 00 0b 53 55 42 51 43 41 43 48 45 53 5a 00 00 02 31 30 00 00 00 00 00 0c
< 2026/05/04 11:07:57.329285 length=2 from=292 to=293
00 0c
> 2026/05/04 11:07:57.329306 length=18 from=446 to=463
00 24 00 09 73 79 73 6d 61 73 74 65 72 00 00 00 00 0c
< 2026/05/04 11:07:57.329462 length=28 from=294 to=321
00 0f 00 15 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:07:57.329505 length=64 from=464 to=527
00 02 00 00 00 00 00 32 43 52 45 41 54 45 20 54 45 4d 50 20 54 41 42 4c 45 20 74 20 28 69 64 20 49 4e 54 45 47 45 52 2c 20 6e 61 6d 65 20 56 41 52 43 48 41 52 28 35 30 29 29 00 16 00 31 00 0c
< 2026/05/04 11:07:57.329630 length=46 from=322 to=367
00 08 00 2d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:07:57.329772 length=8 from=528 to=535
00 04 00 00 00 07 00 0c
< 2026/05/04 11:07:57.335023 length=28 from=368 to=395
00 0f 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:07:57.335068 length=8 from=536 to=543
00 04 00 00 00 0b 00 0c
< 2026/05/04 11:07:57.335119 length=2 from=396 to=397
00 0c
> 2026/05/04 11:07:57.335149 length=42 from=544 to=585
00 02 00 01 00 00 00 1b 49 4e 53 45 52 54 20 49 4e 54 4f 20 74 20 56 41 4c 55 45 53 20 28 31 2c 20 3f 29 00 00 16 00 31 00 0c
< 2026/05/04 11:07:57.335248 length=102 from=398 to=499
00 08 00 06 00 00 00 00 00 00 00 33 00 01 00 00 00 05 00 00 00 00 00 00 00 00 00 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 32 6e 61 6d 65 00 00 00 5e 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:07:57.335370 length=22 from=586 to=607
00 04 00 00 00 05 00 01 00 00 00 00 00 00 00 01 78 00 00 07 00 0c
< 2026/05/04 11:07:57.337281 length=48 from=500 to=547
00 5e 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 00 00 00 00 01 00 00 01 01 00 00 00 00 00 37 00 00 00 01 00 00 00 02 00 0c
> 2026/05/04 11:07:57.337328 length=8 from=608 to=615
00 04 00 00 00 0b 00 0c
< 2026/05/04 11:07:57.337363 length=2 from=548 to=549
00 0c
> 2026/05/04 11:07:57.337382 length=42 from=616 to=657
00 02 00 01 00 00 00 1b 49 4e 53 45 52 54 20 49 4e 54 4f 20 74 20 56 41 4c 55 45 53 20 28 32 2c 20 3f 29 00 00 16 00 31 00 0c
< 2026/05/04 11:07:57.337449 length=102 from=550 to=651
00 08 00 06 00 00 00 00 00 00 00 33 00 01 00 00 00 05 00 00 00 00 00 00 00 00 00 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 32 6e 61 6d 65 00 00 00 5e 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:07:57.337547 length=20 from=658 to=677
00 04 00 00 00 05 00 01 00 00 00 00 00 00 00 00 00 07 00 0c
< 2026/05/04 11:07:57.339301 length=48 from=652 to=699
00 5e 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 00 00 00 00 01 00 00 01 02 00 00 00 00 00 37 00 00 00 01 00 00 00 02 00 0c
> 2026/05/04 11:07:57.339387 length=8 from=678 to=685
00 04 00 00 00 0b 00 0c
< 2026/05/04 11:07:57.339436 length=2 from=700 to=701
00 0c
> 2026/05/04 11:07:57.339475 length=42 from=686 to=727
00 02 00 01 00 00 00 1b 49 4e 53 45 52 54 20 49 4e 54 4f 20 74 20 56 41 4c 55 45 53 20 28 33 2c 20 3f 29 00 00 16 00 31 00 0c
< 2026/05/04 11:07:57.339550 length=102 from=702 to=803
00 08 00 06 00 00 00 00 00 00 00 33 00 01 00 00 00 05 00 00 00 00 00 00 00 00 00 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 32 6e 61 6d 65 00 00 00 5e 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:07:57.339651 length=18 from=728 to=745
00 04 00 00 00 05 00 01 00 00 ff ff 00 00 00 07 00 0c
< 2026/05/04 11:07:57.341388 length=48 from=804 to=851
00 5e 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 00 00 00 00 01 00 00 01 03 00 00 00 00 00 37 00 00 00 02 00 00 00 02 00 0c
> 2026/05/04 11:07:57.341436 length=8 from=746 to=753
00 04 00 00 00 0b 00 0c
< 2026/05/04 11:07:57.341477 length=2 from=852 to=853
00 0c
> 2026/05/04 11:07:57.341493 length=48 from=754 to=801
00 02 00 00 00 00 00 22 53 45 4c 45 43 54 20 69 64 2c 20 6e 61 6d 65 20 46 52 4f 4d 20 74 20 4f 52 44 45 52 20 42 59 20 69 64 00 16 00 31 00 0c
< 2026/05/04 11:07:57.341613 length=114 from=854 to=967
00 08 00 02 00 00 00 00 00 00 00 37 00 02 00 00 00 08 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 00 00 00 03 00 00 00 04 00 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 32 69 64 00 6e 61 6d 65 00 00 0f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 03 00 00 00 02 00 0c
> 2026/05/04 11:07:57.341728 length=42 from=802 to=843
00 04 00 00 00 03 00 12 5f 69 66 78 63 30 30 30 30 30 30 30 30 30 30 30 30 31 00 06 00 04 00 00 00 09 00 00 10 00 00 00 00 0c
< 2026/05/04 11:07:57.341824 length=70 from=968 to=1037
00 0e 00 00 00 00 00 06 00 00 00 01 01 78 00 0e 00 00 00 00 00 05 00 00 00 02 00 00 00 0e 00 00 00 00 00 06 00 00 00 03 01 00 00 0f 00 00 00 00 00 03 00 00 01 03 00 00 00 00 00 37 00 00 00 03 00 00 00 02 00 0c
> 2026/05/04 11:07:57.341923 length=14 from=844 to=857
00 04 00 00 00 09 00 00 10 00 00 00 00 0c
< 2026/05/04 11:07:57.341954 length=28 from=1038 to=1065
00 0f 00 00 00 00 00 03 00 00 01 03 00 00 00 00 00 37 00 00 00 03 00 00 00 02 00 0c
> 2026/05/04 11:07:57.341987 length=8 from=858 to=865
00 04 00 00 00 0a 00 0c
< 2026/05/04 11:07:57.342013 length=2 from=1066 to=1067
00 0c
> 2026/05/04 11:07:57.342026 length=8 from=866 to=873
00 04 00 00 00 0b 00 0c
< 2026/05/04 11:07:57.342052 length=2 from=1068 to=1069
00 0c
> 2026/05/04 11:07:57.342085 length=2 from=874 to=875
00 38
< 2026/05/04 11:07:57.345084 length=2 from=1070 to=1071
00 38
2026/05/04 11:07:57 socat[837404] N socket 1 (fd 6) is at EOF
2026/05/04 11:07:57 socat[837404] N socket 2 (fd 5) is at EOF
2026/05/04 11:07:57 socat[837404] N exiting with status 0
Reference in New Issue View Git Blame Copy Permalink