Skip to content

Capture reasoning_content from OpenAI-compatible Chat Completions - #1691

Open
CodeWithMoin wants to merge 1 commit into
simonw:mainfrom
CodeWithMoin:openai-chat-reasoning-content
Open

CodeWithMoin wants to merge 1 commit into
simonw:mainfrom
CodeWithMoin:openai-chat-reasoning-content

Conversation

@CodeWithMoin

Copy link
Copy Markdown

Refs #1689

DeepSeek, vLLM and llama.cpp (with --reasoning-format deepseek) return reasoning as reasoning_content, on choices[].message or on each streamed choices[].delta. Chat and AsyncChat ignored it, so it never showed on stderr or made it into llm logs.

Now both emit it as StreamEvent(type="reasoning") before the text, streaming and non-streaming, so it goes through the same path as Responses API reasoning: dim on stderr, hidden with -R, stored as a ReasoningPart, shown under ## Reasoning in llm logs and as reasoning in --json.

One related tweak: DeepSeek also reports completion_tokens_details.reasoning_tokens, which used to add the empty redacted=True marker. That marker is now skipped when real reasoning text was emitted, otherwise the response ends up with two reasoning parts. The count-only case (OpenAI o-series) behaves as before.

Reasoning isn't sent back on later turns of a conversation, since _append_llm_message still skips ReasoningPart for Chat. That matches DeepSeek's guidance not to return reasoning_content.

Testing

New TestReasoningContent in tests/test_openai_messages.py: streamed, non-streamed, async streamed, and streamed with reasoning_tokens in usage. All four fail on main.

python -m pytest        # 1128 passed
black --check . / ruff check . / mypy llm   # clean
cog --check docs        # clean

Also ran the CLI against a local fake server streaming reasoning_content, configured through extra-openai-models.yaml like in the issue:

$ llm -m qwen38 'what is 2+2'
Two plus two is four.      <- stderr, dim
4

$ llm logs -n 1
## Reasoning
Two plus two is four.
## Response
4

$ llm logs -n 1 --json   ->   "reasoning": "Two plus two is four."

On main the same run logs "reasoning": null. With --no-stream the reasoning is logged but not printed, same as every other model, since that path only prints response.text().

DeepSeek, vLLM and llama.cpp (--reasoning-format deepseek) return reasoning
as reasoning_content on the message, or on each delta when streaming. The
Chat and AsyncChat models ignored it, so it was never shown on stderr or
stored in llm logs.

Emit it as reasoning StreamEvents, and skip the redacted reasoning marker
when real reasoning text was already emitted.

Refs simonw#1689
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant