Skip to content

fix(openai-ws): never reuse a socket with an abandoned response - #200

Merged
stippi merged 1 commit into
mainfrom
fix/ws-unfinished-response
Sep 15, 2026
Merged

stippi merged 1 commit into
mainfrom
fix/ws-unfinished-response

Conversation

@stippi

@stippi stippi commented Sep 15, 2026

Copy link
Copy Markdown
Owner

Problem

The Responses WebSocket client carries at most one response per socket and has no per-event correlation to a request. When a caller abandons a request mid-stream by dropping the future (a caller that aborts the request task on a user interrupt, say), no error path runs in the client: the connection survives and the server keeps streaming the dead response into it.

The next response.create on that socket then reads the previous response's remaining events as its own answer. From there on every reply is shifted by one request, and the previous_response_id bookkeeping no longer matches the server, until the server rejects the chain with previous_response_not_found and the connection is finally dropped.

Observed in a real session: three previous_response_not_found failures over two hours, each preceded by an interrupt, with the assistant answering the previous message instead of the current one in between.

Fix

  • The connection is marked busy before a request frame goes out; only the response's terminal event (completed, failed, incomplete) clears the mark. ensure_connection retires a marked socket instead of reusing it. This covers the task-abort case, where no code of the client runs.
  • Output events arriving before this request's own response.created fail the request as stale rather than being consumed. Lifecycle events are not affected.
  • previous_response_not_found and a stale stream are retried once on a fresh connection, which resets the chain so the full input is sent.

process_ws_stream now reads from the connection's receiver directly and reports what it established about the response, which makes the reader testable with a scripted event flow.

Tests

  • completed response leaves the connection clean
  • callback abort leaves the response unfinished
  • failed response is terminal
  • output before response.created is rejected as stale and retryable
  • lifecycle events before response.created are not stale
  • rejected incremental chain is retried, other 400s stay final

cargo test -p llm: 129 passed. Clippy clean.

A request abandoned mid-stream (the caller drops the future, e.g. a task
abort on user interrupt) leaves the server streaming a response nobody
reads. The next request on the same socket consumed that response's tail
as its own answer, shifting every later reply by one, until the server
rejected the incremental chain with previous_response_not_found.

- Mark the connection busy before a request goes out; only the response's
  terminal event clears it. ensure_connection retires a marked socket.
- Output events before this request's response.created fail the request
  as stale instead of being consumed.
- A rejected chain or a stale stream is retried once on a fresh
  connection with the full input.
@stippi
stippi merged commit 02a86c1 into main Sep 15, 2026
5 checks passed
@stippi
stippi deleted the fix/ws-unfinished-response branch September 15, 2026 20:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant