Summary
On the non-streaming OpenAI path, models whose parse_nstream_content does not
split the think block deliver the reasoning inside content instead of
reasoning_content, so the client shows raw <think> tags as the answer. When
generation is truncated mid-thought, the partial thought becomes the entire answer.
Affects qwen3.5, qwen3.6-moe, qwen3vl and qwen3.5-omni — all
reasoning-capable. Reproduced on main at 92f3f13.
Environment
- OS: Ubuntu 26.04.1 LTS (kernel 7.0.0-31-generic)
- NPU driver: NPU FW 1.1.2.64, amdxdna 0.7
flm version: FLM v1.0.6
flm validate:
[Linux] Kernel: 7.0.0-31-generic
[Linux] NPU: /dev/accel/accel0 with 8 columns
[Linux] NPU FW Version: 1.1.2.64
[Linux] amdxdna version: 0.7
[Linux] Memlock Limit: infinity
Reproduce
flm serve qwen3.5:9b
curl -s localhost:52625/v1/chat/completions -H 'Content-Type: application/json' -d '{
"model": "qwen3.5:9b",
"messages": [{"role":"user","content":"What is 5 plus 6? Think briefly."}],
"max_tokens": 400, "think": true
}' | jq '.choices[0].message'
Observed on 92f3f13 — no reasoning_content key, thought in content:
finish=stop reasoning_content=False <think>_in_content=True
content: '<think>\nThinking Process:\n\n1. **Analyze the Request:**\n * '
Same with "max_tokens": 60 (finish=length), where the truncated thought is
the whole answer.
Cause
build_nstream_response takes reasoning_content from parse_nstream_content,
and only some models populate it:
| splits reasoning |
does not |
| gemma4_12b, gemma4e, gpt_oss, lfm2, llama3, nanbeige, qwen3 |
qwen3_5vl, qwen3_6_moe, qwen3vl, qwen3_5_omni |
The streaming path is unaffected: its parser switches to REASONING on
<think> and stays there, so streamed responses split correctly. Only the
non-streaming path is wrong.
Impact
Agent clients render the thought as the assistant's reply. A truncated thought
reads as the prompt being echoed back ("Thinking Process: 1. Analyze the
Request: The user wants me to ..."), which looks like the model looping.
Fix
PR follows: split the block once in build_nstream_response so every model
reports reasoning the same way, and route an unclosed (truncated) block to
reasoning_content with empty content. No-op for models that already
populate it.
Summary
On the non-streaming OpenAI path, models whose
parse_nstream_contentdoes notsplit the think block deliver the reasoning inside
contentinstead ofreasoning_content, so the client shows raw<think>tags as the answer. Whengeneration is truncated mid-thought, the partial thought becomes the entire answer.
Affects
qwen3.5,qwen3.6-moe,qwen3vlandqwen3.5-omni— allreasoning-capable. Reproduced on
mainat 92f3f13.Environment
flm version:FLM v1.0.6flm validate:Reproduce
Observed on 92f3f13 — no
reasoning_contentkey, thought incontent:Same with
"max_tokens": 60(finish=length), where the truncated thought isthe whole answer.
Cause
build_nstream_responsetakesreasoning_contentfromparse_nstream_content,and only some models populate it:
The streaming path is unaffected: its parser switches to
REASONINGon<think>and stays there, so streamed responses split correctly. Only thenon-streaming path is wrong.
Impact
Agent clients render the thought as the assistant's reply. A truncated thought
reads as the prompt being echoed back ("Thinking Process: 1. Analyze the
Request: The user wants me to ..."), which looks like the model looping.
Fix
PR follows: split the block once in
build_nstream_responseso every modelreports reasoning the same way, and route an unclosed (truncated) block to
reasoning_contentwith emptycontent. No-op for models that alreadypopulate it.