Capture reasoning_content from OpenAI-compatible Chat Completions - #1691
Open
CodeWithMoin wants to merge 1 commit into
Open
CodeWithMoin wants to merge 1 commit into
CodeWithMoin wants to merge 1 commit into
Conversation
DeepSeek, vLLM and llama.cpp (--reasoning-format deepseek) return reasoning as reasoning_content on the message, or on each delta when streaming. The Chat and AsyncChat models ignored it, so it was never shown on stderr or stored in llm logs. Emit it as reasoning StreamEvents, and skip the redacted reasoning marker when real reasoning text was already emitted. Refs simonw#1689
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #1689
DeepSeek, vLLM and llama.cpp (with
--reasoning-format deepseek) return reasoning asreasoning_content, onchoices[].messageor on each streamedchoices[].delta.ChatandAsyncChatignored it, so it never showed on stderr or made it intollm logs.Now both emit it as
StreamEvent(type="reasoning")before the text, streaming and non-streaming, so it goes through the same path as Responses API reasoning: dim on stderr, hidden with-R, stored as aReasoningPart, shown under## Reasoninginllm logsand asreasoningin--json.One related tweak: DeepSeek also reports
completion_tokens_details.reasoning_tokens, which used to add the emptyredacted=Truemarker. That marker is now skipped when real reasoning text was emitted, otherwise the response ends up with two reasoning parts. The count-only case (OpenAI o-series) behaves as before.Reasoning isn't sent back on later turns of a conversation, since
_append_llm_messagestill skipsReasoningPartfor Chat. That matches DeepSeek's guidance not to returnreasoning_content.Testing
New
TestReasoningContentintests/test_openai_messages.py: streamed, non-streamed, async streamed, and streamed withreasoning_tokensin usage. All four fail onmain.Also ran the CLI against a local fake server streaming
reasoning_content, configured throughextra-openai-models.yamllike in the issue:On
mainthe same run logs"reasoning": null. With--no-streamthe reasoning is logged but not printed, same as every other model, since that path only printsresponse.text().