docs(reasoning): document max_tokens budgeting for reasoning models

[dparkmit24]

docs(reasoning): document max_tokens budgeting for reasoning models

Problem

Several providers now return reasoning by default with no thinking or reasoning_effort in the request (DeepSeek, Moonshot AI, Z.AI; current Gemini models also default to thinking). Reasoning tokens count toward the completion budget, so a modest max_tokens can be consumed entirely by hidden reasoning: the call returns finish_reason: "length" with content empty or nearly empty, and the request is billed for the full completion. From the response alone this reads as ordinary truncation, which makes it a confusing first-contact failure with reasoning models.

I hit this building a model-comparison app on litellm. moonshot/kimi-k2.5 at max_tokens=200 returned finish_reason: "length", content: '', and completion_tokens_details.reasoning_tokens: 199 of 200, billed; the identical request at a higher cap finished cleanly with 623 reasoning plus 17 visible tokens. The same pattern reproduced on Z.AI glm models and on gemini/gemini-3.5-flash (112 of 116 completion tokens consumed by reasoning at a 120 token cap). litellm itself handles all of this correctly; reasoning_content and reasoning_tokens are surfaced cleanly, which is exactly what makes the failure detectable.

The behavior is technically implied by the max_completion_tokens definition on the input params page ("including visible output tokens and reasoning tokens"), and the health check page acknowledges it for probes ("providers count reasoning tokens toward the completion budget"), but the trap itself and a detection recipe are not documented anywhere user-facing.

What this adds

One section on the existing reasoning content page, "Budgeting max_tokens for reasoning models": the mechanism, the failure signature, detection via usage.completion_tokens_details.reasoning_tokens for non-streaming calls and stream_options={"include_usage": True} with the final-chunk usage for streams, and sizing guidance, with links to the existing Streaming Usage page, the Gemini reasoning_effort mapping (including its note that thinking cannot be fully disabled on Gemini 3), and the health check reasoning defaults.

Why scoped here

The reasoning content page is where users land when a reasoning model misbehaves, so the budget interaction belongs there rather than on a new page. No sidebar change. No provider page is touched; the open zai (#530) and vertex (#509) docs PRs do not overlap this change.

Verified with npm run build (passes; the three cross-page links resolve in the built HTML).

Disclosure: built with my two-agent Claude workflow: Claude Code implemented against my written spec, verified with npm run build (Docusaurus link check) and automated review; the finding, the live reproductions, the scope decisions, and the verification standards are mine.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Jul 14, 2026 6:38pm