feat(proxy): keep a Claude session working on a fallback model when Anthropic flags a request

[marcelstenkewitz]

What this changes for users

I built this for my own Clodex setup: when Anthropic flags a request, I wanted the current Claude Code session to keep working through one of my configured fallback models instead of stopping. I figured I would throw the generic mechanism into a PR in case it is useful upstream. It is disabled unless a user configures fallback rules, and it ships with no rules or provider defaults. Enabling a rule means the conversation Anthropic refused — the whole payload, unmodified — is sent to the fallback provider the user configured. Clodex does not enforce that provider's retention or training policy.

Problem and root cause

In proxy mode, a normal Claude-model request is passed through to Anthropic as raw bytes. Anthropic can return HTTP 200 and finish the stream with stop_reason: "refusal"; Clodex observed usage beside that stream but had no refusal hook, so Claude Code received the refusal and the session remained on the same model.

The path is reachable in a normal clodex claude session using Anthropic passthrough plus a configured OpenAI-compatible route. Historical Claude Code transcript rows show both timing shapes: refusals before content, one after a token, and others after 190–6,180 output tokens. That requires two behaviors: retry the current request only before content, but remember every refusal for later requests.

The change

• Adds an optional, first-match flagFallback rule list in ~/.clodex/config.json. A route may be a saved alias or a full Clodex route id. No rules are supplied by default.
• Holds only a matching Anthropic SSE response until its first content block, a refusal, or 64 KiB of pre-content events. A normal response is then released unchanged.
• Retries a pre-content refusal through the configured route without exposing the refusal bytes (tier 2). A refusal after content preserves the current response and moves only later requests (tier 1).
• Remembers the fallback by Claude session id and requested model, bounded to 256 pairs. Later streaming, non-streaming, and count_tokens requests use the remembered route. /clear changes the session id; a proxy restart clears the in-memory map.
• Keeps the request body and requested model unchanged. The selected route travels only on the authenticated local adapter hop, and responses continue to echo the requested model.
• Writes a bounded flag_switch lifecycle row with the tier, category, model, route, and unavailable-route state.
• Keeps the pure matching, SSE classification, and bounded memory in a separate 73-line module instead of further expanding the MITM server.

Deliberately left out: refusal detection in non-streaming response bodies, Anthropic 4xx policy errors, and count_tokens responses; persisted switch state; a CLI for flagFallback; fallback providers or prices; automatic return to Anthropic; and any default routing policy. No rules or provider defaults ship, and the example route names are illustrative.

This is larger than a focused fix and I did not open an issue first. I am happy to close or reshape it if this is not a direction Clodex wants.

Evidence

• pnpm typecheck && pnpm test && pnpm build — Node 24.14.1, pnpm 10.34.5, fresh CLODEX_HOME, proxy and Anthropic override variables unset: 125 test files passed / 2 skipped; 2,930 tests passed / 20 skipped; build succeeded.
• Feature-deletion mutation run — with every source change reset to current main while retaining the six complete changed test files, 16 tests failed and 83 passed. Failures covered rule loading/validation, route override, tier 1, tier 2, session memory, unavailable routes, fallback errors, token counts, gzip, the 64 KiB ceiling, and route-log capping.
• Guard mutations — removing the request-error hold release changed the protected partial 200 into a 502; removing the ceiling or its tier transition and reverting the route cap each turned its intended test red.
• Reachability — the installed build carrying this exact patch handled two real pre-content Fable refusals. Both wrote a tier-2 flag_switch row, retried through a configured Kimi route, returned HTTP 200, and completed with output. The refusal categories were cyber and reasoning_extraction. CLODEX_LOG_REQUEST_PREVIEW was unset.
• Manual smoke — the installed same-patch 2.15.0 build returned exact sentinels through normal Anthropic Haiku passthrough and a direct Kimi/OpenRouter route after the current gate. The run also printed the pre-existing stale-patch warning and Kimi's unrecognized_model notice before returning output.
• Concurrency — after a fresh fetch, origin/main remains f1c74ac; git merge-tree is clean; open PR fix(models): let you raise the context window on a model whose provider publishes none #259 touches context-window and registry files but no file in this diff, so there is no land-order constraint.
• Commit summary line reads as a release note for a non-technical user.

Automated server tests use a fake Anthropic origin and fake fallback adapter. The real checks above cover tier 2 and normal traffic, not a real mid-stream tier-1 refusal, a provider timeout, Windows, or a process restart followed by --resume. CI must supply the Windows leg. Earlier acceptance also ran the full gate under Node 22.19.0; the current pre-PR gate used the repository-pinned Node 24.14.1.

Failure and rollback behavior

A rule naming an unavailable route passes the original refusal through and records unavailable: true. A selected fallback's error reaches the client; the flagged content is not sent back to Anthropic. Malformed saved rules stop proxy startup with a configuration error rather than silently changing routing.

The feature persists no new state and needs no migration. Removing flagFallback (or making it empty) restores ordinary pass-through behavior; restarting the proxy clears remembered switches. Installing an upstream build without this change also removes the feature while leaving the unknown config key inert.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UE5y6RAFZ6Jqk33sk3EKrq