Responses stream: emit response.failed on abnormal termination - #1639
Open
andreasfoo wants to merge 1 commit into
Open
andreasfoo wants to merge 1 commit into
andreasfoo wants to merge 1 commit into
Conversation
…see why
Codex reports every gateway-side stream failure as the same opaque line —
"stream disconnected before completion: stream closed before response.completed"
— and retrying never helps, because the reason we do emit never reaches it.
The gateway already detects both failure shapes (upstream clean EOF with no
terminal event, upstream read/decode error) and emits an `event: error` frame.
But codex-rs's SSE reader (codex-api/src/sse/responses.rs, process_responses_event)
has no match arm for "error": the frame falls into the catch-all and is dropped,
the stream then ends normally, and the client falls back to its generic message
with the actual cause discarded. `response.failed` IS handled there — Codex reads
response.error.{code,message}, classifies it (context window / quota / rate limit
/ retryable) and surfaces the message.
So terminate such streams the way the protocol does: emit `response.failed`
first, then keep the legacy `error` frame for clients that key off it. Codes
are unchanged (upstream_truncated / stream_failed), and the response id already
announced on the wire is carried over so the failure attaches to the response
the client was streaming.
Also fixed, both found while tracing that path:
- `"usage": null` on response.created / response.in_progress was rewritten into
`{"output_tokens_details":{"reasoning_tokens":0}}` by the reasoning-token
backfill, because gjson reports an explicit null as existing. That handed
clients a usage object missing every required field. Require an object.
- openai-go aborts a stream on any event carrying a top-level "error" key —
including `"error": null` — with the gjson string form of that value as the
message, i.e. empty. describeResponsesStreamError re-attaches the raw event so
the log and the client get something to act on instead of a bare
"received error while streaming: ".
- A Responses input item whose `type` the pinned SDK does not model makes the
whole input union decode to nothing, without an error: the request would go
upstream carrying only instructions and tools, with the entire conversation
silently gone — and agentic clients replay their history every turn, so it
repeats for the rest of the session. checkResponsesInputSurvived rejects that
and names the offending item index and type.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ShwPKYgqy6nsZfXT5vwBoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Responses streams that terminate abnormally (upstream truncation, read errors, in-band errors) now emit a
response.failedevent before the legacyerrorframe. This ensures Codex and other strict consumers can diagnose the failure reason instead of reporting a generic "stream closed before response.completed".Key Changes
Responses stream failure signaling:
HandleOpenAIResponsesStreamnow callsSendResponsesStreamFailureon abnormal termination, emittingresponse.failedwith error code and message, followed by the legacyerrorframe for backward compatibility. The response ID is carried through if already announced to the client.Error detail recovery:
describeResponsesStreamErrorrestores diagnostic information that the OpenAI SDK drops when aborting on in-band error events (e.g.,"error": null), re-attaching the raw event to the error message.Input validation for Responses:
checkResponsesInputSurvivedguards against silent input array loss during SDK deserialization. When an item type the pinned SDK doesn't model appears, the entire array decodes to nothing without error — this now fails the request with a clear message naming the offending item, preventing silent conversation loss on retry loops.Usage backfill safety: The reasoning_tokens backfill now checks
IsObject()instead ofExists()to avoid synthesizing incomplete usage objects fromnullvalues inresponse.created/response.in_progressevents.Minor
.design/stream-converter-pipeline.mdwith the Responses stream termination protocol and rationale.Notes
The
response.failedevent structure follows the protocol's own failure format:response.error.{code,message}with codesupstream_truncated(no terminal event) andstream_failed(read/decode error). Codex's SSE reader has a match arm forresponse.failedbut not for bareerrorframes, making this change essential for error visibility in that consumer.https://claude.ai/code/session_01ShwPKYgqy6nsZfXT5vwBoc