fix(token-spy): preserve upstream streaming status - #2970
Open
tang-vu wants to merge 1 commit into
Open
Conversation
This was referenced Aug 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this matters
Token Spy created its downstream
StreamingResponsebefore opening the upstream stream. FastAPI therefore committed HTTP 200 even when Anthropic or an OpenAI-compatible provider returned 4xx/5xx. Clients could treat rate limits and provider failures as successful requests, defeating normal retry and error handling.Root cause: the upstream status was only available inside the response-body iterator, after downstream headers had already been selected.
Behavioral invariant: both supported streaming APIs return the exact upstream HTTP status and still close the upstream response after passthrough.
Overlap check
Searched open and closed PRs for Token Spy streaming/status/error/passthrough terms and inspected PRs changing
ods/extensions/services/token-spy/main.py. PR #1963 improves streaming error-body logging but does not propagate the upstream status; other current Token Spy PRs cover SSE cursors, pricing, writes, and exception handling. This change is independent and does not alter the logging payload.What changed
build_request/send(stream=True)API before creating the downstream response.StreamingResponsestatus from the opened upstream response for Anthropic and OpenAI-compatible routes.Validation
/v1/messagesand/v1/chat/completionsreturned HTTP 200 for an upstream 429.pytest tests/test_streaming_status.py -q— 2 passed.pytest tests -q— 22 passed, 1 skipped.python -m py_compile main.py tests/test_streaming_status.py— passed.git diff --check— passed.Tradeoffs and rollback
The proxy now waits for upstream response headers before sending downstream headers; that is required to preserve status and does not buffer the response body. SSE body processing and usage accounting remain unchanged. Rollback is one commit with no state or schema migration.
Batch compatibility
Validated as an independent ten-PR batch from upstream
main6ff9b4fc5190099705043acaab7e9b6ad9c8b8f1. The final PR heads merged without conflicts in this order: #2964 -> #2965 -> #2967 -> #2969 -> #2970 -> #2971 -> #2972 -> #2973 -> #2974 -> #2975. The resulting local synthetic merge head is4ae60eadad9696a9accb735aad71af87d2be802d.Combined validation on that exact tree:
git diff --check: passed.The scopes are behaviorally independent. The stated order is the tested rollback/merge sequence for shared-file changes; each PR remains individually useful and revertible.