[Bug] Claude Code requests fail on Azure Opus when thinking parameters are not normalized per target #239

Activity

1. xinyu-dev

   changed the title [-]Claude Code requests fail on Azure Opus when thinking parameters are not normalized per target[/-] [+][Bug] Claude Code requests fail on Azure Opus when thinking parameters are not normalized per target[/+] on Aug 3, 2026
2. fedecia commented on Aug 17, 2026

   

   fedecia

   on Aug 17, 2026

   Contributor

   More actions

   Hit this from the opposite direction, and found a config-level workaround that needs no upstream change.

   Same error, mirrored

   Your case: Claude Code sends thinking.type=enabled, Azure Opus 5 requires adaptive + output_config.effort.

   My case: Claude Code sends thinking.type=adaptive and output_config.effort=medium already — exactly what the error message asks for — and Switchyard's translation layer turns it back into enabled before it reaches Bedrock Opus 5. Same error string, generated rather than passed through:

   400 bedrock error: The model returned the following errors:
   "thinking.type.enabled" is not supported for this model.
   Use "thinking.type.adaptive" and "output_config.effort" to control thinking behavior.
   

   So this is not only "Switchyard doesn't normalize an inbound value". A conforming inbound request is rewritten into a non-conforming one. That widens the bug: a client can't avoid it by sending the right thing.

   Environment

   • switchyard-server 0.2.0 (crates.io, the only published version)
   • macOS 26.6, Claude Code as the client, stage_router (not random_routing)
   • Upstream: a PortKey gateway deployment → Bedrock, global.anthropic.claude-opus-5
   • Efficient tier DeepSeek-V4-Flash, capable tier Opus 5

   Different gateway, different route type, different provider — same defect, which suggests it sits in the translation layer rather than in any one provider adapter.

   The trigger is output_config, not the inbound thinking value

   Bisected against one captured Claude Code body (115 tools, thinking={"type":"adaptive","display":"omitted"}, output_config={"effort":"medium"}), replayed through the router:

   Request	HTTP	Result
   unmodified	400	"thinking.type.enabled" is not supported
   thinking removed, output_config kept	400	same error
   output_config removed, thinking kept	200	Opus 5 answers
   both removed	200	Opus 5 answers

   Removing thinking alone does not help. Removing output_config does. So the thinking: {"type": "enabled"} is synthesized from output_config.effort during translation, independent of what the client sent.

   Control: the identical body sent directly to the same gateway and model — bypassing Switchyard — returns 200, streaming and non-streaming, with and without anthropic-beta: effort-2025-11-24. The payload is valid; only the proxied path fails.

   What the debug trace shows

   RUST_LOG=debug names the cause on the failing leg:

   libsy.upstream_attempt{model="global.anthropic.claude-opus-5" wire_format=openai_chat ...}
   libsy.client_call{... gen_ai.request.max_tokens=32000
                         gen_ai.request.reasoning.level="medium"
                         switchyard.routing.tier="strong" ...}
   

   Two things stand out:

   • wire_format=openai_chat — an Anthropic inbound request (/v1/messages) is being encoded as OpenAI Chat for the upstream, because that leg's client declares format = "openai_chat".
   • gen_ai.request.reasoning.level="medium" — output_config.effort has become an OpenAI-style reasoning level. Re-encoding that back to Anthropic for Bedrock is where thinking.type=enabled appears.

   Anthropic → OpenAI IR → Anthropic is not round-trip safe for the effort/thinking pair.

   Workaround: declare the capable leg anthropic_messages

   Declaring the Anthropic upstream with its native wire format skips the translation entirely:

   [llm_clients.portkey_bedrock]
   format = "anthropic_messages"   # was: openai_chat
   base_url = "https://<portkey-gateway>/v1"

   With that one change, the unmodified Claude Code body returns 200:

   x-model-router-selected-model: global.anthropic.claude-opus-5
   x-model-router-rationale: fall-through selected global.anthropic.claude-opus-5 (confidence 0.762)
   

   A full stage_router run then interleaves correctly with total_errors: 0 — cheap turns on DeepSeek, escalated turns on Opus 5, and de-escalation afterwards.

   Your repro config uses format: openai for both targets, so this may clear your Azure leg too — worth trying before waiting on a fix. It only applies when the upstream genuinely speaks Anthropic Messages; a mixed-provider route still needs per-target normalization, which is what this issue asks for.

   One caveat for anyone taking the workaround on a gateway: switching that client to anthropic_messages changes how api_key_env is sent (as x-api-key). A gateway expecting its own auth header will start returning 401 — for PortKey the key has to move to extra_headers as x-portkey-api-key. Related: #360, #419, #420.

   Confirming your observation about the summary counters

   the per-model req value reports successful calls only, while failed attempts appear only in the aggregate error count

   This bit me too, and it is worth fixing alongside. GET /v1/stats showed calls for the capable target while every one of those calls had 400'd, and the tiers array was empty, so nothing indicated the escalation had failed. I initially concluded the router was escalating successfully. The only reliable per-turn signals were the x-model-router-rationale header and the per-model errors count.
3. harshitwandhare commented on Sep 19, 2026

   

   harshitwandhare

   on Sep 19, 2026

   More actions

   I reproduced the translation step for this offline against main at bfcd023c, inside switchyard-translation with no network, and the second root cause in @fedecia's comment does not hold. Switchyard is not turning adaptive back into enabled.

   Feeding in the body shape described there:

   {"model": "claude-opus-5", "messages": [{"role": "user", "content": "hi"}],
    "max_tokens": 32000,
    "thinking": {"type": "adaptive", "display": "omitted"},
    "output_config": {"effort": "medium"}}

   AnthropicMessages -> OpenAiChat produces:

   {"model": "claude-opus-5", "messages": [{"role": "user", "content": "hi"}],
    "max_completion_tokens": 32000, "reasoning_effort": "medium"}

   No thinking key at all, and the string enabled never appears. Translating that result back the other way gives "thinking": {"type": "adaptive"}, still not enabled.

   The mechanism: codecs/anthropic/buffered.rs:74 decodes reasoning from output_config.effort and nothing else. The inbound thinking object is stored at :80 as ReasoningParams.raw, and the Chat encoder never reads it back. The Anthropic encoder at :311 writes {"type": "adaptive"}, and only when effort is set.

   That fits the bisect table in the report better than the translation-layer conclusion does. Removing output_config returned 200 because it removes the only reasoning field Switchyard forwards to Chat. Removing thinking alone changed nothing because Switchyard was already ignoring it. So whatever emits thinking.type=enabled sits downstream, which in that setup is the PortKey gateway in front of Bedrock. Worth checking the body PortKey sends rather than the one Switchyard sends.

   Something adjacent is a real defect, and I have sent #792 for it. Anthropic to Responses does not drop thinking, it copies the object verbatim into the Responses reasoning field:

   in : "thinking": {"type": "enabled", "budget_tokens": 10000}
   out: "reasoning": {"type": "enabled", "budget_tokens": 10000}
   diagnostics: []
   

   type and budget_tokens are not keys the Responses reasoning object defines, and nothing is reported, including under LossyConversionPolicy::Reject. That is closer to the title of this issue than the Chat path is, though it is not the path in the original report.

   Happy to test a patch on the normalization question here if you decide which way it should go.