llm: fix Anthropic thinking/effort payload to match the real API - #100
llm: fix Anthropic thinking/effort payload to match the real API#100sandeep wants to merge 1 commit into
Conversation
thinking:{type:"adaptive"} and output_config:{effort:...} both looked
plausible but aren't real fields in Anthropic's public Messages API. Found
live against claude-sonnet-4-5 (the flagship model): the real API rejects
"adaptive" outright ("adaptive thinking is not supported on this model")
and rejects output_config.effort on every model tried ("This model does
not support the effort parameter"). shellm always passes --thinking
--effort together, so every shellm call to a real Claude model failed
100% of the time — chat replies (a separate, simpler direct llm call with
no --thinking) were unaffected, which is why this stayed hidden.
- thinking now sends type:"enabled" with a required budget_tokens (half of
max_tokens, floored at the API's 1024 minimum) instead of "adaptive".
- effort is now gated behind a capability guard (supports_anthropic_effort,
mirroring supports_anthropic_thinking) that returns false for every
known model, since none of them accept it — LLM_ASSUME_THINKING=1 is the
same escape hatch already used for the thinking guards.
New tests/test_llm_anthropic_thinking.sh covers both fixes plus the escape
hatch, with curl stubbed to capture the actual outgoing payload.
nickjalbert
left a comment
There was a problem hiding this comment.
Thanks for the PR! From review:
The patch breaks Claude Opus 4.7 and new Anthropic models
bin/llm:467 treats every current Claude family alike. bin/llm:521 then sends thinking.type: "enabled" with budget_tokens, while bin/llm:483 drops output_config.effort for every model.
That payload fixes Sonnet 4.5, but Claude 4.7 and later reject manual extended thinking and require adaptive thinking with effort. Headlong defaults to Opus 4.7 and always passes both flags, so normal shellm calls would fail. Anthropic also documents effort support for several current models. Anthropic's API primer describes the model split.
Add an explicit model capability table. At minimum, test Sonnet 4.5 with enabled plus a budget, and Opus 4.7 with adaptive plus effort. Include the supported 4.6 behavior as well.
Small max_tokens values produce an invalid budget.
bin/llm:527 floors the thinking budget at 1024 without ensuring it remains below max_tokens. The new test therefore accepts max_tokens: 100 with budget_tokens: 1024.
Anthropic requires budget_tokens to be at least 1024 and less than max_tokens. Anthropic Messages API reference.
Reject this combination, disable thinking with a warning, or raise the effective output limit. Add boundary tests for 1024 and 1025 tokens.
After those are addressed and you rebase on main, we should be good to 🚢 !
Fixes #99
What
thinking:{type:"adaptive"}andoutput_config:{effort:...}are not realfields in Anthropic's public Messages API. The real API rejects both on
claude-sonnet-4-5, its flagship model.
shellmalways callsbin/llmwith
--thinking --effort high, so monolith wakeup fails on any realAnthropic Claude model.
respondercallsbin/llmdirectly, without--thinking/--effort, so chat replies keep working and can look likeeverything is fine.
How it works
thinking.typeis now"enabled", with a requiredbudget_tokens—half of
max_tokens, floored at the API's 1024 minimum, leaving theother half of the token budget for the visible response.
output_config.effortis now gated behind a newsupports_anthropic_effort()guard, mirroring the existingsupports_anthropic_thinking()/supports_openai_reasoning()/supports_gemini_thinking()guards. It returns false for every knownmodel, since none of them accept this field via the real API.
LLM_ASSUME_THINKING=1is the same escape hatch already used for theother guards, for a future model that does support it.
--thinking, the Anthropic section under "Thinking &effort") updated to match.
Tests
New
tests/test_llm_anthropic_thinking.sh(9 checks, curl stubbed tocapture the outgoing payload):
thinking.typeis"enabled"not"adaptive",budget_tokensis half ofmax_tokensand floors at 1024,--effortno longer addsoutput_configand is reported on stderr,thinkingstill sends when--effortis dropped alongside it (the exactcombination
shellmalways sends), andLLM_ASSUME_THINKING=1forcesoutput_config.effortthrough.Verified against the real Anthropic API (not just the stub): both flags
together on
claude-sonnet-4-5now succeed end to end, live on apreviously-broken identity — monolith went from 100%
rc=1to a normalmulti-iteration run immediately after.
Full
tests/run-all.sh: 37/38 pass; the one failure(
test_thinkers_pending.sh) is unrelated dispatcher-queue timing, andfails the same way on unmodified
mainin this environment.shellcheck -S warning,bash -n, and a bash-3.2 parse check are allclean.