fix: GPT-5.6 SSE streams truncated through /v1/responses (chunk framing, raw identity passthrough, gzip window)

[Jiliac]

Problem

chatgpt/gpt-5.6-{sol,terra,luna} (ChatGPT-subscription Codex backend) return HTTP 200 through Plano's /v1/responses, but the client only receives response.created + response.in_progress — no text, no response.completed. Fixes #980.

Live debugging with a from-source gateway traced this to three stacked defects, each independently able to truncate or corrupt streams:

1. SSE chunk-boundary loss in SseChunkProcessor (hermesllm)

SseStreamIter splits the buffer with str::lines() and parses each line independently. When a network chunk boundary lands inside a data: /event: line prefix (e.g. da + ta: {…}), the partial line fails with a non-JSON error, which SseStreamIter::next silently skips — it never reaches the incomplete-JSON buffering. Both halves are unparseable, so the event is permanently lost with no log line. A boundary inside a multi-byte UTF-8 char could similarly kill the whole chunk via str::from_utf8.

Fix: byte-level SSE framing in process_chunk — only bytes up to the last \n are parsed; the trailing partial line is held back and prepended to the next chunk (with a completeness check for a final unterminated line like data: [DONE], since nothing flushes the buffer at end-of-stream, and a 1 MiB cap so a broken upstream can't grow the buffer unboundedly).

2. Lossy parse-and-reconstruct on the identity Responses→Responses path

The "passthrough" path strictly deserialized every event into ResponsesAPIStreamEvent and re-serialized from the struct. Any unmodeled event type or field was dropped or mangled: via-Plano streams lost reasoning.mode/reasoning.context, text.verbosity, tool_usage, …, and a future unknown event type (e.g. response.reasoning_text.delta) would be dropped entirely. Additionally, llm_gateway's per-event loop aborted the whole remaining chunk (return Err(Action::Continue)) when one event failed to parse.

Fix: on the identity Responses→Responses path, forward the original wire bytes verbatim (reconstructing the coupled event: <type>\ndata: <json>\n\n block from the raw payload — output is byte-identical to upstream). Parsing is still attempted for token counting; unknown events forward raw with provider_stream_response = None, and incomplete-JSON errors still propagate for cross-chunk recombination (shared is_incomplete_json_error helper, used by both call sites). stream_context.rs now logs and falls through on unparsed events instead of aborting the chunk. Cross-API arms and Anthropic/ChatCompletions identity behavior are untouched.

3. gzip decompressor window_bits: 9 truncates streams (envoy config)

This turned out to be the dominant user-visible cause. Envoy's decompressor filter advertises accept-encoding upstream, so the backend returns gzip. The decompressor was configured with window_bits: 9 (512-byte inflate window) while the paired compressor uses window_bits: 10 and real upstreams use 15. Gzip headers don't carry the window size, so inflate doesn't fail upfront — it silently stops emitting data once a back-reference exceeds the window. Debug trace of a failing stream: response data decompressed from 1372 bytes to 2638 bytes, then 307 → 2, then 0 for every remaining frame. This truncated any sufficiently large/redundant SSE stream traversing two listeners (gpt-5.4 included, when the client path involved compression).

Fix: decompressor window_bits: 15 on all four gzip decompressor blocks (three were at 9; the egress_traffic_llm one had no window_bits and silently used Envoy's default of 12). Note the blast radius: this config applies to all providers' gzip-encoded responses, not just chatgpt — which is intended, since the truncation class affects any of them. Raising the inflate window is strictly safe (a larger window decodes anything a smaller one can; max_inflate_ratio zip-bomb protection is retained).

Also: added gpt-5.6-{sol,terra,luna} to the chatgpt models in provider_models.yaml (consumed via include_str! in ProviderId), and a pre-existing one-line clippy fix (for_kv_map) needed to keep the workspace -D warnings gate green.

Verification

Unit tests (all inline, ids obfuscated):

• test_gpt56_responses_identity_passthrough_full_stream — replays a real captured GPT-5.6 stream (11 events incl. a reasoning output item) through the processor at every 64-byte split offset, asserting all event types emerge and output bytes == input bytes. Failed before the fix; passes now.
• Mid-prefix split, mid-JSON recombination, unknown-event tolerance, bounded-buffer, and serde relaxation tests (OutputItem::Reasoning without summary / with content/encrypted_content/status; omitted logprobs).
• Full gates: cargo test --lib (388 tests across brightstaff/common/hermesllm/llm_gateway/prompt_gateway), cargo fmt --check, cargo clippy --locked --all-targets --all-features -- -D warnings, wasm32-wasip1 release builds.

Live through a from-source gateway (planoai build + up):

• All three chatgpt/gpt-5.6-* tiers and chatgpt/gpt-5.4 now stream the full ladder (created … output_text.delta … completed) with correct answer text.
• Previously-dropped fields (reasoning.mode, text.verbosity, tool_usage) are preserved byte-for-byte.

Notes

• Wire-format change: identity Responses→Responses streams are now verbatim upstream bytes instead of Plano's re-serialized shape — strictly more spec-faithful; scoped to that identity pair only.
• Orthogonal to Stoff81/chatgpt5.5 #958 (ChatGPT/5.5 path): this change deliberately doesn't touch the ResponsesAPIStreamBuffer cross-API path Stoff81/chatgpt5.5 #958 works in.

🤖 Generated with Claude Code

[Spherrrical]

Reviewed the code and ran the full local gate suite against this branch — cargo test --lib (176 tests across the 5 crates), cargo fmt --check, cargo clippy --locked --all-targets --all-features -D warnings, and the wasm32-wasip1 release build all pass, including the 7 new regression tests. Nice work; the every-64-byte-offset replay test is a great guard.

A few non-blocking hardening/clarity asks below. Two things I couldn't cover locally and am trusting the from-source verification for: the live gzip window_bits behavior (needs a running Envoy + real upstream) and the e2e streaming path.

Also, on the window_bits: 15 change — just flagging for ops awareness that this now affects gzip decoding for all providers, not only chatgpt. That's the correct scope given the truncation class, but it's not isolated to the reported model.

Nit: the full-stream test asserts out.len() == baseline_len + substring presence, but the PR description claims byte-identical output. Consider a strict assert_eq!(out, expected_bytes) if you want the test to actually enforce that claim.

[Spherrrical]

A chunk boundary landing inside an event: line (e.g. event: respo) parses as a valid event-only SseEvent, so None => true makes it "complete" and it gets consumed now with a truncated name; the continuation on the next chunk is then dropped as unparseable. This is safe today only because every coupled path re-derives the event name from data.type and suppresses the raw event: line — an implicit invariant, not an enforced one.

Could you either hold back a trailing event:-only line too (return false here when the line isn't newline-terminated), or add a comment documenting why truncating an event: line is safe, so a future non-coupled consumer doesn't silently regress?

[Jiliac]

Fixed in 59e5428. Trailing event-only lines without a newline are now always retained, so a split such as event: respo cannot be consumed as a complete event. Added test_mid_event_name_split_is_buffered_until_continuation, which asserts both buffering after the first fragment and exact recombination after its continuation.

[Spherrrical]

Could you add a unit test pinning the actual serde_json EOF messages this must catch (so a wording change fails loudly), or classify on serde_json::Error::is_eof() where the concrete type is still available rather than the stringified Box<dyn Error>?

[Jiliac]

Fixed in 59e5428. Internal streaming retry decisions now downcast to serde_json::Error and use is_eof() rather than matching rendered error text. The public Display-based helper remains for API compatibility. Regression coverage confirms truncated JSON is retried, invalid JSON is not, and an unrelated UnexpectedEof is not misclassified.

[Jiliac]

Addressed the remaining review items in 59e5428 and pushed the branch. The identity-stream fixture now asserts byte-for-byte equality for the single-chunk baseline and every 64-byte boundary phase. The gzip window_bits: 15 all-provider scope remains documented in the PR description; it is intentionally global to gzip-encoded traffic through the affected listeners. Fresh verification passed: cargo fmt --all -- --check, workspace Clippy with warnings denied, cargo test --lib, and the wasm32-wasip1 release build for both gateway plugins.

[Jiliac]

thanks @Spherrrical 🙏