fix(providers): deliver the reasoning budget on OpenRouter, and never drop a param in silence - #369
Conversation
… drop a param in silence `reasoning_effort` was configured and discarded. litellm's openrouter transformation only adds the param to its supported list when `supports_reasoning()` is true — i.e. when the model is in litellm's capability map — and with `drop_params = True` an unmapped model loses it with no warning. The models a reasoning budget is set for are exactly the ones the map does not know yet. Measured on this repo: one lens on #367 spent 34,012 reasoning tokens against a 32,768 ceiling with `reasoning_effort: low` set, and truncated. Two changes: - `dropped_params` reads litellm's own capability map and OpenAI-param vocabulary to name, once at startup, every configured param the resolved model will not accept. Keyed off the map rather than a per-param special case, so a param added later is covered; provider-native options (ollama's `num_ctx`) are not judged, since they are not litellm's to drop. - On openrouter only, a budget litellm will not forward is sent as OpenRouter's own top-level `reasoning` object via `extra_body`. Never beside the flat param — OpenRouter answers a request carrying both with a 400. `default` has no equivalent in OpenRouter's effort enum, so it is reported, not translated. The openrouter branch is a deliberate, narrowly scoped exception to the litellm-normalises-everything decision in CLAUDE.md, documented as such in the code — not licence for general per-provider plumbing. Also corrects `.lgtmaybe.yml`, whose comments concluded `reasoning_effort` was "the separate lever" against a knob that was never connected. The measured table stays; the conclusion drawn from it does not. Refs #348 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
max_tokens ceiling on reasoning, so the batch was not split — a smaller payload cannot shrink a thinking budget; lower reasoning_effort instead). Check the provider credentials/quota, model, and timeout (ollama: a larger model needs a longer --timeout), then retry.
lgtmaybe 1.12.2
|
lgtmaybe 1.12.2 |
Closes the loop on #348.
reasoning_efforthas been set in this repo's.lgtmaybe.ymlsince 1.12.0 and was never reaching the model.The bug
cli/__init__.pyputsreasoning_effortinto the provider opts correctly and ungated. The adapter setslitellm.drop_params = True(so one unsupported param can't fail a whole review). litellm's openrouter transformation addsreasoning_effortto its supported-params list only whenlitellm.supports_reasoning(model=...)is true — i.e. only for models already in litellm's capability map. Everything else is dropped with no warning.Re-verified against the installed litellm (1.94.1), reading
OpenrouterConfig.get_supported_openai_paramsand callingget_optional_paramsdirectly:supports_reasoningopenrouter/~deepseek/deepseek-v4-flash-latestopenrouter/deepseek/deepseek-v4-flash-latestopenrouter/deepseek/deepseek-r1openrouter/anthropic/claude-sonnet-4.5It is not the
~floating-alias prefix — the bare name is unmapped too. The models a reasoning budget gets configured for are precisely the newest ones, which are precisely the ones the map does not know yet.Live consequence: #367's review, with
reasoning_effort: lowset in this repo's config, still had a lens spend 34,012 reasoning tokens against the 32,768max_tokensceiling and truncate.Part 1 — never silently discard a configured param
providers/factory.dropped_paramsreads litellm's own two maps —get_supported_openai_paramsfor the resolved model, andOPENAI_CHAT_COMPLETION_PARAMSfor the vocabulary — and names every param the user configured that the model will not accept. One structured warning at startup, next in spirit to the existing "per-call timeout resolved" line:It runs in the factory rather than the CLI because that is where the litellm model string the capability map keys on comes into existence, and it judges exactly the user-configured opts — before the factory adds its own timeout, credentials, and ollama options.
Deliberately general, deliberately small: keyed off the capability map, so a param added to
ReviewConfiglater is covered without touching this function. Only OpenAI-vocabulary params are judged — a provider-native passthrough like ollama'snum_ctxnever appears in the map and is not litellm's to drop, so flagging it would only train the reader to ignore the warning. A lookup failure reports nothing; this is instrumentation and must never be why a review doesn't run.Part 2 — actually deliver the budget on OpenRouter
Verified against OpenRouter's current documentation (
openrouter.ai/docs/guides/best-practices/reasoning-tokensand theChatRequestReasoningEffortschema in its API reference, via Context7 — the docs site 403s direct fetches):reasoningobject:{"effort": ..., "max_tokens": ..., "exclude": ..., "enabled": ...}reasoning.effortacceptsxhigh|high|medium|low|minimal|nonereasoning_effortexists as a flat shorthand, and the docs state it "cannot be used simultaneously with reasoning.effort if they differ"So on the openrouter route only, when litellm has been observed to drop the flat param, the budget goes out as
extra_body={"reasoning": {"effort": ...}}instead.No double-send. OpenRouter rejects a request carrying both with
400 Only one of "reasoning" and "reasoning_effort" may be provided. Theextra_bodyis added only on the branch wheredropped_paramsalready reported the flat param dropped, and the flat param is popped when it is. When litellm will forward natively (thedeepseek-r1case), nothing is injected.Value mapping.
ReviewConfig.reasoning_effortis litellm's normalised set:none | minimal | low | medium | high | xhigh | default. Six of the seven are in OpenRouter's enum verbatim and pass through unchanged.defaulthas no equivalent — it is omitted and reported through Part 1's warning rather than translated into a nearby level, which would quietly buy a budget nobody asked for. (litellm's own openrouter transformation translatingmax→xhighis the hint that this is their vocabulary, not OpenAI's.)Scope. Every other route is byte-identical — asserted directly (
test_no_other_route_is_reshaped), and the whole existing provider matrix still passes unchanged.A deliberate exception, not a precedent
CLAUDE.md's "Key decisions (do not relitigate)" names litellm as the provider spine that "normalises … to one
completion()call". This is a maintainer-approved exception to that, for one param on one route, because litellm's normalisation is keyed on a model list that structurally lags the models people point at. It is called out as such in_honour_param_support's docstring and in the spec. It is not licence for general per-provider plumbing.Part 3 — correcting the record
.lgtmaybe.yml's comments concludedreasoning_effortwas "the separate lever" and told the reader to step up tomediumif findings looked shallow. That conclusion was reached against a knob that was never connected. The measured table stays — it is real, and it is still the evidence thatmax_tokensis the wrong lever — but the conclusion drawn from it is corrected, with a note to re-measure before moving the value now that the budget is actually enforced.docs/how-to/reduce-review-cost.mdgains a short subsection on the OpenRouter path and the new warning line;docs/llms-full.txtregenerated.Verification — what is genuinely verified vs mock-asserted
Genuinely verified against real litellm (no mocks):
openrouter/vendor/unmapped-modeldropsreasoning_effortandopenai/gpt-4odrops it — both read from litellm's real capability map at test timenum_ctx/thinkare never reported as dropped (realOPENAI_CHAT_COMPLETION_PARAMS)logprobsonanthropic/claude-3-haikuis reported, and nobody wrote a case for itreasoningobject survives litellm's own openrouter transformation —test_the_native_field_survives_litellms_own_transformationruns the built opts through the reallitellm.get_optional_params. This matters:OpenrouterConfig.map_openai_paramsassignsmapped_openai_params["extra_body"]for its owntransforms/models/routeparams, so a future bump that made that assignment clobber rather than merge would put us straight back to a budget that looks sent and never leaves. The test catches that.Asserted against a mock:
litellm.completionand asserts the kwargs handed to it (the existingtests/cli/test_provider_threading.pyseam) — real CLI, real config load, real factory, real engine, fake transportget_supported_openai_paramsrather than naming a specific model, so the no-double-send guarantee can't rot into a false pass when a model leaves litellm's map on a dependency bumpNot verified: no live OpenRouter call was made (no key). OpenRouter's acceptance of the field is from its published documentation, not from a 200 response. The wire format is documented in two independent places in their docs and corroborated by the 400 error text quoted above.
test_reasoning_effort_flag_reaches_litellmpreviously asserted the flat param reached litellm onopenrouter/vendor/m— i.e. it asserted the broken behaviour and passed because it stopped one layer above where the drop happens. It now points at ollama, where litellm's map really does forward it.Coordination
Stayed entirely out of
providers/litellm_provider.py— this is request/param assembly, in the factory. #368 owns the response-mapping region (ProviderTruncated,reasoning_tokens) and its truncation message. Worth noting: that message namesreasoning_effortas the lever to reach for, which becomes accurate once this lands — today it points at a knob that does nothing on the route this repo actually runs on.Gate
uv run ruff check .·uv run ruff format --check .·uv run mypy·uv run pytest -q— all green (1838 passed, 3 skipped).uv run pytest tests/specs -qgreen;openspec validate --specsgreen. Spec: one new requirement underprovider-gatewaywith aprovider.param-supportanchor bound todropped_params.🤖 Generated with Claude Code
Generated by Claude Code