Repository navigation
gateway: a 402 the walk fails over on cools an unfunded key (D258) - #120
Merged
Merged
Conversation
jaredLunde
enabled auto-merge (squash)
October 4, 2026 15:40
… (D258) A walk fails over on a 402 at the response head (D84), so Redact never read the body D180 cools an out-of-credit key from: every later request paid a round trip to the unfunded provider before failing over. Pingora's hooks cannot read an abandoned body, so the status decides: a 402 is out of credit by its status alone on every provider but OpenRouter, whose 402 may be one request too costly for the balance (remedy::unfunded_402). upstream_response_filter cools that key on both abandoning paths (in-place retry, recorded FullBody retry). logging no longer cools from an abandoned FullBody attempt's body, which raced the parent's drop and would double-count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
jaredLunde
force-pushed
the
d258-402-failover-cooling
branch
from
October 4, 2026 15:42
709e696 to
1d51bcf
Compare
jaredLunde
added a commit
that referenced
this pull request
Oct 4, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by the full mutation run's triage (#118).
Bug. A managed walk fails over on a candidate's 402 at the response head (D84), so the body is never read and D180's out-of-credit cooling never runs. With a funded backup on the row, every later request first made a round trip to the unfunded provider. Reproduced on main: Anthropic
billing_error402 first, all 3 requests hit Anthropic,ai_key_auth_failures_total{reason="unfunded"}stayed 0 (small body and 100 KiB FullBody path alike).Fix. Pingora 0.9 gives no hook to read an abandoned attempt's body, so decide by status where status is enough:
remedy::unfunded_402— a 402 is out-of-credit on every provider except OpenRouter (whose 402 can mean "this request requires more credits", which must not cool a shared key; D180/D200). A failing-over managed 402 cools the key inupstream_response_filteron both drop paths (in-place retry, recorded FullBody retry), via the existingProvider::mark_key_badcooldown and a sharedcool_unfunded_keyhelper. No 2xx-path work, no new state.loggingno longer cools from an abandoned FullBody attempt's body (it could double-count).Still open (documented in D258 and ARCHITECTURE.md): an OpenRouter 402 the walk fails over on still cools nothing; by default OpenRouter is last in the walk (D253), where its 402 is relayed and cooled from the body.
Tests:
a_402_the_walk_fails_over_on_cools_an_unfunded_key(small + 100 KiB; Anthropic 402 → hit once, key cooled; OpenRouter too-costly 402 → not cooled),remedy::tests::only_openrouter_needs_the_body_of_a_402.🤖 Generated with Claude Code
https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk