Repository navigation
gateway: a run of OpenRouter 402s cools its key when OpenRouter leads the walk (D261) - #122
Merged
Merged
Conversation
jaredLunde
enabled auto-merge (squash)
October 4, 2026 16:36
D258 cools a 402 a catalog walk fails over on by its status alone, but not OpenRouter's, which may be one request too costly for a funded balance; a walk that leads with OpenRouter paid a round trip to an unfunded account on every request. An out-of-band balance check cannot tell the cases apart (GET /api/v1/key reports only a per-key limit, null for a key drawing on account credit; GET /api/v1/credits needs a management key). Each unread OpenRouter 402 is a strike against its pool key (a relaxed AtomicU32 beside the key's cooldown); the third in a row with no 2xx from the key between them cools it through cool_unfunded_key (same metric reason, same warn line). Any 2xx resets the count (one relaxed load, a store only when non-zero), any cooling spends it, and a key already cooling does not count, so a burst cools it once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
Kills the changed-lines survivor that made any candidate refusal (401/403) a strike. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
jaredLunde
force-pushed
the
openrouter-402-cooling
branch
from
October 4, 2026 16:52
a914073 to
d830e80
Compare
jaredLunde
added a commit
that referenced
this pull request
Oct 4, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the gap D258 left: when a row or
x-beyond-orderputs OpenRouter first, its 402 fails over at the head (body unread), and OpenRouter's 402 is ambiguous (out of credit vs. "this request requires more credits"), so the unfunded key was never cooled and every later request paid a round trip to it.OpenRouter's API can't settle it with an inference key (
/api/v1/keyis null for keys without their own limit;/api/v1/creditsneeds a management key), so this uses a repeat heuristic instead:FullBodyre-run) is a strike against that configured pool key. The 3rd in a row with no 2xx from that key in between cools it via the existingcool_unfunded_key→Provider::mark_key_bad(metric reasonunfunded).AtomicU32beside each pool key's existing cooldown slot. Per replica, in memory, bounded by configured keys; BYO keys are never touched.KEY_COOLDOWN(60 s).Tests:
reliability_openrouter_402(3 in a row cool, small + 100 KiB bodies; too-costly 402s between successes never cool; 24 concurrent 402s cool once) androute::testsfor per-key isolation, recovery after cooldown, and cooling spending the count. 11/12 hand mutations caught, 1 equivalent (documented). D261 in verify/defects.toml; ARCHITECTURE.md updated.🤖 Generated with Claude Code
https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk