warehack.ing/informix-db
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 6.b: DATETIME decoding for all qualifier ranges

Browse Source 
Before:
  cur.execute("SELECT CURRENT YEAR TO SECOND ...")
  cur.fetchone()  # → (b'\xc7\x14\x1a\x05\x04...',) raw BCD bytes

After:
  cur.execute("SELECT CURRENT YEAR TO SECOND ...")
  cur.fetchone()  # → (datetime.datetime(2026, 5, 4, 12, 34, 56),)

Decoder picks the right Python type by qualifier:
  YEAR/MONTH/DAY-only → datetime.date
  HOUR/MIN/SEC-only   → datetime.time
  spans across both   → datetime.datetime

Wire format (per IfxToJavaDateTime + Decimal.init treating as packed BCD):
  byte[0] = sign + biased exponent (in base-100 digit pairs)
  byte[1..] = BCD digit pairs: YYYY (2 bytes) + MM + DD + HH + MI + SS + FFFFF

Qualifier extraction from column descriptor:
  encoded_length = (digit_count << 8) | (start_TU << 4) | end_TU
  TU codes: YEAR=0, MONTH=2, DAY=4, HOUR=6, MIN=8, SEC=10,
            FRAC1=11..FRAC5=15

Verified against four DATETIME columns of different qualifiers in
one tuple — see test_datetime_multiple_columns_in_one_row:
  YEAR TO SECOND       → datetime.datetime(2026, 5, 4, 12, 34, 56)
  YEAR TO DAY          → datetime.date(2026, 5, 4)
  HOUR TO SECOND       → datetime.time(12, 34, 56)
  YEAR TO FRACTION(3)  → datetime.datetime(...)

Module changes:
  src/informix_db/converters.py:
    + _decode_datetime(raw, encoded_length) — qualifier-driven BCD walk
    + TU constants (_TU_YEAR, _TU_MONTH, ..., _TU_SECOND)
  src/informix_db/_resultset.py:
    + DATETIME row-decoder branch — computes width from digit_count
      in encoded_length high byte, calls _decode_datetime with the
      packed qualifier so it can pick the right Python type

Tests: 40 unit + 70 integration (7 new DATETIME tests) = 110 total,
all green, ruff clean. Tests cover:
  - YEAR TO SECOND → datetime.datetime
  - YEAR TO DAY → datetime.date
  - HOUR TO SECOND → datetime.time
  - CURRENT YEAR TO FRACTION(3) → datetime.datetime
  - Mixed qualifiers in one row
  - DATETIME stored in a real table column (round-trip via SELECT)
  - NULL DATETIME → Python None

DATETIME parameter binding (encoder) is Phase 6.x — same status as
DECIMAL encoder.
This commit is contained in:
Ryan Malloy 2026-05-04 12:02:40 -06:00
parent d04000dfc3
commit 6819dd4cb0
5 changed files with 315 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
docs/CAPTURES/23-py-datetime.socat.log Normal file
View File

@ -0,0 +1,50 @@
2026/05/04 11:59:34 socat[952456] N listening on AF=2 0.0.0.0:9090
2026/05/04 11:59:34 socat[952456] N accepting connection from AF=2 127.0.0.1:46568 on AF=2 127.0.0.1:9090
2026/05/04 11:59:34 socat[952456] N opening connection to 127.0.0.1:9088
2026/05/04 11:59:34 socat[952456] N opening connection to AF=2 127.0.0.1:9088
2026/05/04 11:59:34 socat[952456] N successfully connected from local address AF=2 127.0.0.1:50314
2026/05/04 11:59:34 socat[952456] N successfully connected to 127.0.0.1:9088
2026/05/04 11:59:34 socat[952456] N starting data transfer loop with FDs [6,6] and [5,5]
> 2026/05/04 11:59:34.462494 length=384 from=0 to=383
01 80 01 3c 00 00 00 64 00 65 00 00 00 3d 00 06 49 45 45 45 4d 00 00 6c 73 71 6c 65 78 65 63 00 00 00 00 00 00 06 39 2e 32 38 30 00 00 0c 52 44 53 23 52 30 30 30 30 30 30 00 00 05 73 71 6c 69 00 00 00 01 3c 00 00 00 00 00 00 00 00 00 01 00 09 69 6e 66 6f 72 6d 69 78 00 00 07 69 6e 34 6d 69 78 00 6f 6c 00 00 00 00 00 00 00 00 00 3d 74 6c 69 74 63 70 00 00 00 00 00 01 00 68 00 0b 00 00 00 03 00 09 69 6e 66 6f 72 6d 69 78 00 00 00 00 00 00 00 00 00 00 00 00 6a 00 06 00 07 44 42 50 41 54 48 00 00 02 2e 00 00 0e 43 4c 49 45 4e 54 5f 4c 4f 43 41 4c 45 00 00 0d 65 6e 5f 55 53 2e 38 38 35 39 2d 31 00 00 11 43 4c 4e 54 5f 50 41 4d 5f 43 41 50 41 42 4c 45 00 00 02 31 00 00 07 44 42 44 41 54 45 00 00 06 59 34 4d 44 2d 00 00 0c 49 46 58 5f 55 50 44 44 45 53 43 00 00 02 31 00 00 09 4e 4f 44 45 46 44 41 43 00 00 03 6e 6f 00 00 6b 00 00 00 00 00 0e 88 96 00 00 00 00 00 0b 72 70 6d 2d 62 75 6c 6c 65 74 00 00 00 00 29 2f 68 6f 6d 65 2f 72 70 6d 2f 63 6c 61 75 64 65 2f 69 6e 66 6f 72 6d 69 78 2f 70 79 74 68 6f 6e 2d 6c 69 62 72 61 72 79 00 00 74 00 20 00 00 00 00 00 00 00 00 00 16 69 6e 66 6f 72 6d 69 78 2d 64 62 40 70 69 64 39 35 32 34 37 30 00 00 7f
< 2026/05/04 11:59:34.474234 length=276 from=0 to=275
01 14 02 3c 10 00 00 64 00 65 00 00 00 3d 00 06 49 45 45 45 49 00 00 6c 73 72 76 69 6e 66 78 00 00 00 00 00 00 2f 49 42 4d 20 49 6e 66 6f 72 6d 69 78 20 44 79 6e 61 6d 69 63 20 53 65 72 76 65 72 20 56 65 72 73 69 6f 6e 20 31 35 2e 30 2e 31 2e 30 2e 33 00 00 07 73 65 72 69 61 6c 00 00 09 69 6e 66 6f 72 6d 69 78 00 00 00 01 3c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 6f 6e 00 00 00 00 00 00 00 00 00 3d 73 6f 63 74 63 70 00 00 00 00 00 00 00 66 00 00 00 00 00 00 00 00 00 00 00 14 00 00 00 6b 00 00 00 00 00 00 03 1a 00 00 00 00 00 0d 32 33 32 37 63 34 33 35 34 65 61 38 00 00 00 00 0f 2f 68 6f 6d 65 2f 69 6e 66 6f 72 6d 69 78 00 00 6e 00 04 00 00 00 00 00 74 00 33 00 00 00 c8 00 00 00 c8 00 29 2f 6f 70 74 2f 69 62 6d 2f 69 6e 66 6f 72 6d 69 78 2f 76 31 35 2e 30 2e 31 2e 30 2e 33 2f 62 69 6e 2f 6f 6e 69 6e 69 74 00 00 7f
> 2026/05/04 11:59:34.474536 length=14 from=384 to=397
00 7e 00 08 ff fc 7f fc 3c 8c aa 97 00 0c
< 2026/05/04 11:59:34.474624 length=16 from=276 to=291
00 7e 00 09 bd be 9f fe 7f b7 ff ef ff 00 00 0c
> 2026/05/04 11:59:34.474664 length=48 from=398 to=445
00 51 00 06 00 26 00 0c 00 04 00 06 44 42 54 45 4d 50 00 04 2f 74 6d 70 00 0b 53 55 42 51 43 41 43 48 45 53 5a 00 00 02 31 30 00 00 00 00 00 0c
< 2026/05/04 11:59:34.474778 length=2 from=292 to=293
00 0c
> 2026/05/04 11:59:34.474798 length=18 from=446 to=463
00 24 00 09 73 79 73 6d 61 73 74 65 72 00 00 00 00 0c
< 2026/05/04 11:59:34.475015 length=28 from=294 to=321
00 0f 00 15 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:59:34.475056 length=210 from=464 to=673
00 02 00 00 00 00 00 c4 53 45 4c 45 43 54 0a 20 20 20 20 44 41 54 45 54 49 4d 45 28 32 30 32 36 2d 30 35 2d 30 34 20 31 32 3a 33 34 3a 35 36 29 20 59 45 41 52 20 54 4f 20 53 45 43 4f 4e 44 2c 0a 20 20 20 20 44 41 54 45 54 49 4d 45 28 32 30 32 36 2d 30 35 2d 30 34 29 20 59 45 41 52 20 54 4f 20 44 41 59 2c 0a 20 20 20 20 44 41 54 45 54 49 4d 45 28 31 32 3a 33 34 3a 35 36 29 20 48 4f 55 52 20 54 4f 20 53 45 43 4f 4e 44 2c 0a 20 20 20 20 43 55 52 52 45 4e 54 20 59 45 41 52 20 54 4f 20 46 52 41 43 54 49 4f 4e 28 33 29 0a 46 52 4f 4d 20 73 79 73 74 61 62 6c 65 73 20 57 48 45 52 45 20 74 61 62 69 64 20 3d 20 31 00 16 00 31 00 0c
< 2026/05/04 11:59:34.475376 length=212 from=322 to=533
00 08 00 02 00 00 00 00 00 00 00 1b 00 04 00 00 00 2e 00 00 00 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0e 0a 00 00 00 0b 00 00 00 08 00 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08 04 00 00 00 16 00 00 00 0d 00 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 06 6a 00 00 00 21 00 00 00 11 00 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 11 0d 28 63 6f 6e 73 74 61 6e 74 29 00 28 63 6f 6e 73 74 61 6e 74 29 00 28 63 6f 6e 73 74 61 6e 74 29 00 28 65 78 70 72 65 73 73 69 6f 6e 29 00 00 0f 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:59:34.475583 length=42 from=674 to=715
00 04 00 00 00 03 00 12 5f 69 66 78 63 30 30 30 30 30 30 30 30 30 30 30 30 31 00 06 00 04 00 00 00 09 00 00 10 00 00 00 00 0c
< 2026/05/04 11:59:34.475676 length=64 from=534 to=597
00 0e 00 00 00 00 00 1b c7 14 1a 05 04 0c 22 38 c7 14 1a 05 04 c3 0c 22 38 c7 14 1a 05 04 11 3b 21 00 00 00 00 0f 00 10 00 00 00 01 00 00 03 01 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:59:34.475748 length=14 from=716 to=729
00 04 00 00 00 09 00 00 10 00 00 00 00 0c
< 2026/05/04 11:59:34.475779 length=28 from=598 to=625
00 0f 00 10 00 00 00 01 00 00 03 01 00 00 00 00 00 37 00 00 00 01 00 00 00 01 00 0c
> 2026/05/04 11:59:34.475813 length=8 from=730 to=737
00 04 00 00 00 0a 00 0c
< 2026/05/04 11:59:34.475843 length=2 from=626 to=627
00 0c
> 2026/05/04 11:59:34.475857 length=8 from=738 to=745
00 04 00 00 00 0b 00 0c
< 2026/05/04 11:59:34.475891 length=2 from=628 to=629
00 0c
> 2026/05/04 11:59:34.475915 length=2 from=746 to=747
00 38
< 2026/05/04 11:59:34.475954 length=2 from=630 to=631
00 38
2026/05/04 11:59:34 socat[952456] N socket 1 (fd 6) is at EOF
2026/05/04 11:59:34 socat[952456] N socket 2 (fd 5) is at EOF
2026/05/04 11:59:34 socat[952456] N exiting with status 0

48
docs/DECISION_LOG.md
View File

@ -355,6 +355,54 @@ literal DECIMAL values, negatives, fractions, NULLs.
---
## 2026-05-04 — Better error messages with PEP 249 exception classification
**Status**: active
**Decision**: ``_raise_sq_err`` decodes the full SQ_ERR payload (sqlcode, isamcode, offset, near-token) and raises the appropriate PEP 249 exception class with a human-readable message and structured fields (``e.sqlcode``, ``e.isamcode``, ``e.offset``, ``e.near``).
PEP 249 classification by sqlcode:
- IntegrityError: -239, -268, -291, -292, -391, -703 (constraint violations)
- ProgrammingError: -201, -206, -217, -286, -310, ... (syntax/object/permission)
- OperationalError: -255, -256, -407, -440, -908, ... (transaction/connection)
- NotSupportedError: -329, -349, -510 (caller-can't-fix)
- DatabaseError: everything else (safe fallback)
Built-in error catalog of ~50 most common Informix sqlcodes in
``src/informix_db/_errcodes.py``. Users extend at runtime via
``register_error_text(code, text)``.
**Connection survives errors**: a failed query doesn't poison the
session — subsequent ``execute()`` calls work normally. Verified by
``test_connection_survives_query_error``.
---
## 2026-05-04 — DATETIME decoding: BCD-packed with qualifier-driven field walk
**Status**: active
**Decision**: ``_decode_datetime(raw, encoded_length)`` walks BCD digit pairs into Python ``datetime`` objects. Returns ``datetime.date`` for date-only qualifiers, ``datetime.time`` for time-only, ``datetime.datetime`` for combined.
Wire format:
- byte[0] = sign + biased exponent (in base-100 digit pairs before decimal)
- byte[1..] = BCD digit pairs (year takes 2 bytes = 4 digits; everything else 1 byte = 2 digits)
The qualifier is packed in the column descriptor's ``encoded_length``:
- high byte = digit_count (total base-10 digits)
- middle nibble = start_TU (time-unit code: YEAR=0, MONTH=2, DAY=4, HOUR=6, MIN=8, SEC=10, FRAC1=11..FRAC5=15)
- low nibble = end_TU
Byte width on the wire = ``ceil(digit_count / 2) + 1``.
Verified against 4 simultaneous DATETIME columns in one tuple:
- YEAR TO SECOND → datetime.datetime(2026, 5, 4, 12, 34, 56)
- YEAR TO DAY → datetime.date(2026, 5, 4)
- HOUR TO SECOND → datetime.time(12, 34, 56)
- YEAR TO FRACTION(3) → datetime.datetime(...)
DATETIME parameter binding (encoder) is Phase 6.x — same status as DECIMAL encoder.
---
## (template — copy below this line for new entries)
```

13
src/informix_db/_resultset.py
View File

@ -257,6 +257,19 @@ def parse_tuple_payload(
values.append(raw)
continue
# DATETIME: width = ceil(digit_count/2) + 1, where digit_count is the
# high byte of encoded_length (packed as (digit_count << 8) |
# (start_TU << 4) | end_TU). The decoder needs the qualifier too,
# so we call it directly here rather than via the dispatch.
if base == int(IfxType.DATETIME):
digit_count = (col.encoded_length >> 8) & 0xFF
width = (digit_count + 1) // 2 + 1
raw = payload[offset:offset + width]
offset += width
from .converters import _decode_datetime
values.append(_decode_datetime(raw, col.encoded_length))
continue
# Fixed-width types
width = FIXED_WIDTHS.get(base)
if width is None:

81
src/informix_db/converters.py
View File

@ -96,6 +96,87 @@ def _decode_date(raw: bytes) -> datetime.date | None:
return _INFORMIX_DATE_EPOCH + datetime.timedelta(days=days)
# Informix DATETIME time-unit (TU) codes — used in qualifier nibbles
_TU_YEAR = 0
_TU_MONTH = 2
_TU_DAY = 4
_TU_HOUR = 6
_TU_MINUTE = 8
_TU_SECOND = 10
# 11-15 are FRACTION(1)..FRACTION(5)
def _decode_datetime(raw: bytes, encoded_length: int) -> object | None:
"""Decode IDS DATETIME bytes per the qualifier in ``encoded_length``.
Wire format per IfxToJavaDateTime + Decimal.init (treating as packed BCD):
byte[0] = sign + biased exponent (in base-100 digit pairs)
byte[1..] = BCD digit pairs, in order: YYYY MM DD HH MI SS FFFFF
The column's ``encoded_length`` packs the qualifier as:
(digit_count << 8) | (start_TU << 4) | end_TU
where TU codes are YEAR=0 / MONTH=2 / DAY=4 / HOUR=6 / MIN=8 /
SEC=10 / FRAC1=11..FRAC5=15.
Returns:
- ``datetime.date`` for qualifiers ending at YEAR/MONTH/DAY
- ``datetime.time`` for qualifiers starting at HOUR or later
- ``datetime.datetime`` for full date+time spans
- ``None`` for NULL (per Decimal NULL marker)
"""
if len(raw) < 2 or (raw[0] == 0 and raw[1] == 0):
return None
start_tu = (encoded_length >> 4) & 0x0F
end_tu = encoded_length & 0x0F
digit_count = (encoded_length >> 8) & 0xFF # noqa: F841 — for sanity
# Decode BCD digit pairs from raw[1..] into a flat string
digit_str = "".join(f"{b:02d}" for b in raw[1:])
# Walk the digit string field-by-field per the qualifier range.
# Each TU consumes a fixed digit count:
# YEAR=4, MONTH=2, DAY=2, HOUR=2, MIN=2, SEC=2, FRACx=2x (x=1..5)
pos = 0
year = month = day = 1
hour = minute = second = 0
microsecond = 0
def consume(n: int) -> int:
nonlocal pos
v = int(digit_str[pos:pos + n])
pos += n
return v
if start_tu <= _TU_YEAR <= end_tu:
year = consume(4)
if start_tu <= _TU_MONTH <= end_tu:
month = consume(2)
if start_tu <= _TU_DAY <= end_tu:
day = consume(2)
if start_tu <= _TU_HOUR <= end_tu:
hour = consume(2)
if start_tu <= _TU_MINUTE <= end_tu:
minute = consume(2)
if start_tu <= _TU_SECOND <= end_tu:
second = consume(2)
if end_tu > _TU_SECOND:
# FRACTION(N) — N digits past the decimal, but stored as N base-100
# digits (= 2*N decimal digits) — we truncate/pad to microseconds.
frac_digits_count = end_tu - _TU_SECOND # 1..5
# Each FRAC unit is 2 BCD digits; total = 2*frac_digits_count chars
frac_str = digit_str[pos:pos + 2 * frac_digits_count].ljust(6, "0")[:6]
microsecond = int(frac_str)
pos += 2 * frac_digits_count
# Pick the right Python type based on what fields are present.
if start_tu >= _TU_HOUR:
return datetime.time(hour, minute, second, microsecond)
if end_tu <= _TU_DAY:
return datetime.date(year, month, day)
return datetime.datetime(year, month, day, hour, minute, second, microsecond)
def _decode_decimal(raw: bytes) -> decimal.Decimal | None:
"""Decode IDS DECIMAL/MONEY: base-100 packed BCD with sign/exponent header.

123
tests/test_datetime.py Normal file
View File

@ -0,0 +1,123 @@
"""Phase 6.b integration tests — DATETIME decoding for all qualifier ranges.
DATETIME is BCD-packed with a qualifier embedded in encoded_length:
``(digit_count << 8) | (start_TU << 4) | end_TU`` where TU codes are
YEAR=0, MONTH=2, DAY=4, HOUR=6, MIN=8, SEC=10, FRAC1=11..FRAC5=15.
The decoder picks the appropriate Python type:
- datetime.date for qualifiers ending at YEAR/MONTH/DAY
- datetime.time for qualifiers starting at HOUR or later
- datetime.datetime for spans crossing the date/time boundary
"""
from __future__ import annotations
import datetime
import pytest
import informix_db
from tests.conftest import ConnParams
pytestmark = pytest.mark.integration
def _connect(conn_params: ConnParams) -> informix_db.Connection:
return informix_db.connect(
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password=conn_params.password,
database=conn_params.database,
server=conn_params.server,
connect_timeout=10.0,
read_timeout=10.0,
)
def test_datetime_year_to_second(conn_params: ConnParams) -> None:
"""Full date+time → datetime.datetime."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute(
"SELECT DATETIME(2026-05-04 12:34:56) YEAR TO SECOND "
"FROM systables WHERE tabid = 1"
)
assert cur.fetchone() == (datetime.datetime(2026, 5, 4, 12, 34, 56),)
def test_datetime_year_to_day(conn_params: ConnParams) -> None:
"""Date-only qualifier → datetime.date."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute(
"SELECT DATETIME(2026-05-04) YEAR TO DAY "
"FROM systables WHERE tabid = 1"
)
assert cur.fetchone() == (datetime.date(2026, 5, 4),)
def test_datetime_hour_to_second(conn_params: ConnParams) -> None:
"""Time-only qualifier → datetime.time."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute(
"SELECT DATETIME(12:34:56) HOUR TO SECOND "
"FROM systables WHERE tabid = 1"
)
assert cur.fetchone() == (datetime.time(12, 34, 56),)
def test_datetime_current_with_fraction(conn_params: ConnParams) -> None:
"""CURRENT YEAR TO FRACTION returns a datetime.datetime."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute(
"SELECT CURRENT YEAR TO FRACTION(3) FROM systables WHERE tabid = 1"
)
(val,) = cur.fetchone()
assert isinstance(val, datetime.datetime)
# Sanity: should be within a reasonable timeframe of "now"
now = datetime.datetime.now()
assert abs((val - now).total_seconds()) < 86400 # within a day
def test_datetime_multiple_columns_in_one_row(conn_params: ConnParams) -> None:
"""Multiple DATETIME qualifiers in one row — proves per-column slicing."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute(
"SELECT "
" DATETIME(2026-05-04 12:34:56) YEAR TO SECOND, "
" DATETIME(2026-05-04) YEAR TO DAY, "
" DATETIME(12:34:56) HOUR TO SECOND "
"FROM systables WHERE tabid = 1"
)
ts, d, t = cur.fetchone()
assert ts == datetime.datetime(2026, 5, 4, 12, 34, 56)
assert d == datetime.date(2026, 5, 4)
assert t == datetime.time(12, 34, 56)
def test_datetime_column_in_table(conn_params: ConnParams) -> None:
"""DATETIME stored in a table column round-trips correctly."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute(
"CREATE TEMP TABLE t_dt (id INTEGER, ts DATETIME YEAR TO SECOND)"
)
cur.execute(
"INSERT INTO t_dt VALUES (1, DATETIME(2026-01-15 09:30:00) YEAR TO SECOND)"
)
cur.execute("SELECT ts FROM t_dt")
assert cur.fetchone() == (datetime.datetime(2026, 1, 15, 9, 30, 0),)
def test_datetime_null(conn_params: ConnParams) -> None:
"""NULL DATETIME decodes to Python None."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("CREATE TEMP TABLE t_dt2 (ts DATETIME YEAR TO SECOND)")
cur.execute("INSERT INTO t_dt2 VALUES (NULL)")
cur.execute("SELECT ts FROM t_dt2")
assert cur.fetchone() == (None,)