Polish item #1: byte-for-byte regression test that asserts our generated login PDU is structurally identical to JDBC's reference captured in docs/CAPTURES/01-connect-only.socat.log. The test (tests/test_pdu_match.py) immediately caught a real bug: the capability section was misread during Phase 0 byte-decoding. Earlier text claimed Cap_1=1, Cap_2=0x3c000000, Cap_3=0 — actually: Cap_1 = 0x0000013c (= (capability_class << 8) | protocol_version where protocol_version = 0x3c = PF_PROT_SQLI_0600) Cap_2 = 0 Cap_3 = 0 The misalignment was: the 0x3c byte I attributed to Cap_2's high byte was actually Cap_1's low byte. The dev-image server is permissive enough to accept arbitrary capability values, so the connection succeeded even with the wrong bytes — but the PDU wasn't structurally identical to JDBC's reference. SERVER-ACCEPTS ≠ STRUCTURALLY-CORRECT. This is exactly why the byte-for-byte diff was the right polish item; "it connects" was a false ceiling. After fix: - 6 PDU-match tests assert byte-for-byte equality at offsets 2..280 (the structural prefix: SLheader sans length, all login markers, capability ints, username, password, protocol IDs, env vars). - Bytes 280+ legitimately differ per process (PID, TID, hostname, cwd, AppName) — those are NOT asserted. - Length field (offsets 0..1) also legitimately differs because our PDU has shorter env list and AppName. - Test uses monkey-patched IfxSocket so no network is needed. Polish item #2: Makefile per global CLAUDE.md convention. Targets: install, lint, format, test, test-integration, test-all, test-pdu, ifx-up/down/logs/shell/status, capture (re-run JDBC scenarios under socat), clean. `make` (no target) prints help. Doc updates: - PROTOCOL_NOTES.md §12: corrected capability section with the actual values and an explanation of the methodology lesson - DECISION_LOG.md: new entry recording the correction with a pointer to the regression test and the takeaway Side artifacts: - docs/CAPTURES/03-py-connect-only.socat.log - docs/CAPTURES/04-py-no-database.socat.log - docs/CAPTURES/05-py-fixed-caps.socat.log Test counts: 40 unit + 6 integration = 46 total, all green, ruff clean.
592 lines
26 KiB
Markdown
592 lines
26 KiB
Markdown
# SQLI Wire Protocol Notes
|
||
|
||
> **Phase 0 spike artifact.** Byte-level reference for the Informix SQLI wire protocol, derived from clean-room study of the decompiled IBM Informix JDBC driver (`ifxjdbc.jar`, `Implementation-Version: 4.50.10-SNAPSHOT` / printable `4.50.JC10`, build 146 from 2023-03-07; SHA256 in `JDBC_NOTES.md`) cross-checked against packet captures of the reference exchange (pending).
|
||
|
||
---
|
||
|
||
## Source attribution conventions
|
||
|
||
Each documented byte sequence cites both sources of evidence:
|
||
|
||
- 🟡 **JDBC**: cross-referenced against `<class>#<method>()` in the decompiled tree (see `JDBC_NOTES.md`)
|
||
- 🔵 **PCAP**: observed in `docs/CAPTURES/<file>.pcap` at offset `<n>`
|
||
- ✅ **CONFIRMED**: corroborated by both 🟡 and 🔵
|
||
- 🟠 **UNVERIFIED**: only one of the two sources
|
||
|
||
**Current state** (2026-05-02): all 🟡 findings are present from JDBC reading. PCAP capture is pending (Phase 0 task #7) and required for ✅. Treat everything below as 🟠 pending PCAP cross-check.
|
||
|
||
---
|
||
|
||
## 1. Wire framing primitives 🟡
|
||
|
||
### Endianness
|
||
|
||
**Big-endian (network byte order) for all multi-byte integers.**
|
||
|
||
Source: `com.informix.lang.JavaToIfxType.JavaToIfxInt(int)` line 51-54:
|
||
```java
|
||
public static byte[] JavaToIfxInt(int i) {
|
||
byte[] b = new byte[]{(byte)(i >> 24), (byte)(i >> 16), (byte)(i >> 8), (byte)i};
|
||
return b;
|
||
}
|
||
```
|
||
Same pattern for `JavaToIfxSmallInt` (2 bytes) and `JavaToIfxLongBigInt` (8 bytes). Java's `DataOutputStream` defaults to big-endian; Informix matches that.
|
||
|
||
### Width table
|
||
|
||
| Wire type | Width (bytes) | Java method | Notes |
|
||
|-----------|---------------|-------------|-------|
|
||
| SmallInt | 2 | `writeSmallInt(short)` / `writeShort` | SMALLINT, message-type tags, length prefixes |
|
||
| Int | 4 | `writeInt(int)` | INTEGER, capability flags, sizes |
|
||
| LongBigInt| 8 | `writeLongBigint(long)` | BIGINT (use this for 64-bit ints) |
|
||
| LongInt | 10 | `writeLongInt(long)` | **Legacy variable-numeric LONG INT — skip MVP**, predates 64-bit ints |
|
||
| Real | 4 | `writeReal(float)` | IEEE 754 single |
|
||
| Double | 8 | `writeDouble(double)` | IEEE 754 double |
|
||
| Date | 4 | `writeDate(Date)` | day-count from Informix epoch (1899-12-31, to confirm) |
|
||
|
||
### Variable-length encoding (string, decimal, datetime, interval, BYTE, TEXT)
|
||
|
||
```
|
||
[short length][bytes payload][optional 0x00 pad if length is odd]
|
||
```
|
||
|
||
**The 16-bit alignment is a hard requirement.** Source: `IfxDataOutputStream.writePadded(byte[])`:
|
||
```java
|
||
public void writePadded(byte[] b) throws IOException {
|
||
this.write(b, 0, b.length);
|
||
if ((b.length & 1) >= 1) {
|
||
this.write(0);
|
||
}
|
||
}
|
||
```
|
||
Mirror in `IfxDataInputStream.readPadded`. Every variable-payload message needs this padding or the next short read will misalign and the entire parser desynchronizes.
|
||
|
||
### String encoding
|
||
|
||
`com.informix.lang.JavaToIfxType.JavaToIfxChar(String)` returns:
|
||
```
|
||
[short length+1][bytes][0x00 nul terminator]
|
||
```
|
||
The `+1` is the trailing nul. So an N-character string takes `2 + N + 1` bytes, then padded to even.
|
||
|
||
Note: there is also a `JavaToIfx4BytesChar` variant for wide-char strings. Probably for GLS multibyte locales. Phase 6+.
|
||
|
||
---
|
||
|
||
## 2. Connection establishment 🟡
|
||
|
||
### TCP setup
|
||
|
||
- Default port: **9088** (native SQLI). Port 9089 is SSL.
|
||
- `Socket.connect(InetSocketAddress(host, port), loginTimeout)`
|
||
- `setTcpNoDelay(true)` — Nagle off
|
||
- `setKeepAlive(socKeepAlive)` — opt-in via `IFX_SOC_KEEPALIVE` property
|
||
- Optional SSL wrap via `SSLSocketFactory`. **Phase 6+; skip for MVP.**
|
||
- Buffered streams: `BufferedInputStream(in, 4096)` / `BufferedOutputStream(out, 4096)` over the raw socket streams. The `IfxDataInputStream`/`IfxDataOutputStream` wrap the buffered streams.
|
||
|
||
### Capability bits (driving login path selection)
|
||
|
||
- `capabilities == 0` → legacy text-mode login (`EncodeAscString`)
|
||
- `capabilities > 0` → modern binary login (`encodeAscBinary`) **— this is what we implement**
|
||
|
||
The capabilities bitfield itself is opaque from the client side; the JDBC driver computes it based on opt-props. For MVP we will hardcode a sensible value matching what JDBC sends in a vanilla connection (will derive from PCAP).
|
||
|
||
### Other constants seen in `Connection.java`
|
||
|
||
| Constant | Value | Notes |
|
||
|----------|-------|-------|
|
||
| `MAX_BUFF_SIZE` | 32768 | upper bound for individual PDUs |
|
||
| `MIN_BUFF_SIZE` | 140 | lower bound |
|
||
| `STREAM_BUF_SIZE` | 4096 | socket buffer size |
|
||
| `PFCONREQ_BUF_SIZE` | 2048 | login request buffer |
|
||
| `SL_HEADER_SIZE` | 6 | the SLheader is 6 bytes |
|
||
| `applType` | `"sqlexec |