fix(qwen35-spec): parse tool calls whenever the request carries tools, independent of the grammar switch by alpineQ · Pull Request #727 · warpfront/hipfire

alpineQ · 2026-09-05T10:12:28Z

Summary

On the speculative (DFlash/MTP) route, tool definitions reached the emitter only when the grammar matcher was armed, so with grammar off the tool-call parser never ran and a well-formed Qwen XML call was streamed to the client as plain text with finish_reason: stop and no calls; tools now always reach the emitter and SpecEmitCtx.grammar carries the matcher switch separately.

What was wrong

generate_dflash (crates/hipfire-generate/src/qwen.rs) built the emit request with

let emit_tools = if grammar_enabled { tools.map(|t| t.to_vec()) } else { None };

Qwen35Emit::new derives both tool_protocol_enabled (arms the XML/JSON tool-call parser) and the grammar ToolSchema list from ctx.tools. Coupling them meant that whenever qwen35_grammar_enabled said no, the parser was disabled too. Measured with an agent harness against qwen3.8-27b (read_file tool, 33k context):

route	tool calls recognised
AR route	2 of 4
speculative route (MTP on)	0 of 2

with the model's text_preview holding a correct <tool_call><function=read_file>…</function></tool_call> every time. For an agent client this is worse than an error: chatter comes back where a call was expected.

What changes

SpecEmitCtx / SpecEmitRequest gain grammar: bool. Qwen35Emit builds the grammar ToolSchema list only when grammar is set; the tool-call parser follows tools alone.
generate_dflash always forwards tools; grammar carries the arch grammar switch. Other constructors (dense.rs, daemon slots, tests) pass grammar: true, i.e. unchanged behaviour.
The router's malformed-body error now logs the failing body prefix ([qwen35-spec] …) instead of silently latching router_malformed.

Which surface(s) does this touch?

kernel
load
serve — crates/hipfire-generate (qwen.rs, dense.rs, common.rs), runtime spec.rs / emit_text.rs, daemon slots.rs
arch crate(s): hipfire-arch-qwen35 (spec_emit.rs)
crates/hipfire-quantize
control plane
docs / CI / scripts only
policy files

Test plan

./scripts/no-gpu-ci.sh: Rust check + no-GPU unit tests + env/docs drift check pass; the Python stage reports 838 passed / 6 failed in tests/test_mq4c_repack.py, and those 6 fail identically on untouched master in this environment (NixOS: no /bin/bash, and the test module resolves a different mq4c_repack than the script) — not touched by this change
cargo build --release clean (--workspace --all-targets --locked)
cargo test --lib --workspace passes; hipfire-generate integration tests (qwen_dflash_semantic_terminal_tests, ds4_malformed_terminal_tests) pass with the new field
serve changes: serve_harness.py battery + chain on qwen3.6:27b (registry fixture) — JSON attached
Tool-call cycle on the speculative route: a 3-turn agent conversation with a read_file tool through the daemon JSONL (qwen3.8-27b.mq4 + .mtp sidecar, MTP K=3, greedy, thinking off): turns 0 and 2 return a parsed calls entry (previously calls: 0 with the XML in text), the tool-result turn continues from the prefix cache. Turn 2 cold-prefills — the MTP drafter has no checkpoint ring on master, unrelated to this change.

turn	finish	tool calls parsed	prompt tokens	cached	prefilled	decode tok/s	text
0	tool_calls	1	2646	0	2646	45.7	``
1	stop	0	2699	2673	26	9.2	`It defines a` main `function that prints the string "hello" to standard output.⏎It defines`
2	tool_calls	1	2754	0	2754	7.6	``

local serve_harness JSON

// daemon md5 a0930e83ad7189584501b1980b48801e (release build of this branch, gfx1100, ROCm 7.2 / HIP 7.2, 2x RX 7900 XTX)
// qwen3.6-27b.mq4 sha256 86a5f80fd29d545abb1093dead242725ced6d68b8607c6d566d897b1a82442dc (registry fixture qwen3.6:27b)
// --- q36-battery.json
[
 {
  "request_id": "chatcmpl-1085773-1",
  "ctx": 34,
  "cached": 0,
  "gen": 132,
  "finish": "stop",
  "think_words": 0,
  "ans_words": 62,
  "prefill_ms": 118.0,
  "prefill_tok_s": 288.1,
  "decode_tok_s": 49.8,
  "decode_estimated": false,
  "tau": null,
  "cycles": null,
  "dflash": null,
  "mtp": null,
  "mtp_ngram": null,
  "ngram_mod_windows": null,
  "ngram_mod_drafts": null,
  "ngram_mod_accepted": null,
  "ngram_mod_accept_rate": null,
  "mtp_windows": null,
  "ar_windows": null,
  "mtp_retired": null,
  "mtp_window_timings": null,
  "ttft_s": 0.13,
  "wall_s": 2.772,
  "attractor": false,
  "empty": false,
  "runaway": false,
  "ans_preview": "```python\ndef longest_substring_without_repeating(s: str) -> str:\n    char_index = {}\n    ",
  "assistant_content": "```python\ndef longest_substring_without_repeating(s: str) -> str:\n    char_index = {}\n    start = 0\n    max_length = 0\n    max_start = 0\n\n    for end, char in enumerate(s):\n        if char in char_index and char_index[char] >= start:\n            start = char_index[char] + 1\n        char_index[char] = end\n        if end - start + 1 > max_length:\n            max_length = end - start + 1\n            max_start = start\n\n    return s[max_start:max_start + max_length]\n```",
  "content": "```python\ndef longest_substring_without_repeating(s: str) -> str:\n    char_index = {}\n    start = 0\n    max_length = 0\n    max_start = 0\n\n    for end, char in enumerate(s):\n        if char in char_index and char_index[char] >= start:\n            start = char_index[char] + 1\n        char_index[char] = end\n        if end - start + 1 > max_length:\n            max_length = end - start + 1\n            max_start = start\n\n    return s[max_start:max_start + max_length]\n```",
  "reasoning_content": "",
  "tool_calls": [],
  "request_md5": "ac7884ced07cabc95cd665752c0ede05",
  "atem_leak": false,
  "prompt_md5": "caf1acdae66df8381a3ce1b0727886ca",
  "expected_substrings": [
   "def "
  ],
  "retrieval_missing": []
 },
 {
  "request_id": "chatcmpl-1085773-3",
  "ctx": 30,
  "cached": 0,
  "gen": 81,
  "finish": "stop",
  "think_words": 0,
  "ans_words": 71,
  "prefill_ms": 71.3,
  "prefill_tok_s": 420.8,
  "decode_tok_s": 50.0,
  "decode_estimated": false,
  "tau": null,
  "cycles": null,
  "dflash": null,
  "mtp": null,
  "mtp_ngram": null,
  "ngram_mod_windows": null,
  "ngram_mod_drafts": null,
  "ngram_mod_accepted": null,
  "ngram_mod_accept_rate": null,
  "mtp_windows": null,
  "ar_windows": null,
  "mtp_retired": null,
  "mtp_window_timings": null,
  "ttft_s": 0.091,
  "wall_s": 1.711,
  "attractor": false,
  "empty": false,
  "runaway": false,
  "ans_preview": "During the day, sunlight passes through less of the atmosphere, allowing shorter blue wave",
  "assistant_content": "During the day, sunlight passes through less of the atmosphere, allowing shorter blue wavelengths to scatter more widely than other colors, which makes the sky appear blue. At sunset, sunlight travels through a thicker layer of atmosphere, causing most of the blue light to scatter away before it reaches your eyes. This leaves the longer red and orange wavelengths to dominate the visual field, creating the vibrant colors seen during twilight.",
  "content": "During the day, sunlight passes through less of the atmosphere, allowing shorter blue wavelengths to scatter more widely than other colors, which makes the sky appear blue. At sunset, sunlight travels through a thicker layer of atmosphere, causing most of the blue light to scatter away before it reaches your eyes. This leaves the longer red and orange wavelengths to dominate the visual field, creating the vibrant colors seen during twilight.",
  "reasoning_content": "",
  "tool_calls": [],
  "request_md5": "f5f1c4d17b48892e03bf90235a7f09e9",
  "atem_leak": false,
  "prompt_md5": "1579a134be91ab67653a176a5cb3007d",
  "expected_substrings": [
   "scatter"
  ],
  "retrieval_missing": []
 },
 {
  "request_id": "chatcmpl-1085773-5",
  "ctx": 31,
  "cached": 0,
  "gen": 19,
  "finish": "stop",
  "think_words": 0,
  "ans_words": 16,
  "prefill_ms": 71.9,
  "prefill_tok_s": 431.5,
  "decode_tok_s": 50.0,
  "decode_estimated": false,
  "tau": null,
  "cycles": null,
  "dflash": null,
  "mtp": null,
  "mtp_ngram": null,
  "ngram_mod_windows": null,
  "ngram_mod_drafts": null,
  "ngram_mod_accepted": null,
  "ngram_mod_accept_rate": null,
  "mtp_windows": null,
  "ar_windows": null,
  "mtp_retired": null,
  "mtp_window_timings": null,
  "ttft_s": 0.092,
  "wall_s": 0.472,
  "attractor": false,
  "empty": false,
  "runaway": false,
  "ans_preview": "The capital of France is Paris, and the river that runs through it is the Seine.",
  "assistant_content": "The capital of France is Paris, and the river that runs through it is the Seine.",
  "content": "The capital of France is Paris, and the river that runs through it is the Seine.",
  "reasoning_content": "",
  "tool_calls": [],
  "request_md5": "ce8f1851f034f316731e0669291e69c2",
  "atem_leak": false,
  "prompt_md5": "8caca580290fb75cea2524f52dd6d094",
  "expected_substrings": [
   "Paris",
   "Seine"
  ],
  "retrieval_missing": []
 },
 {
  "request_id": "chatcmpl-1085773-7",
  "ctx": 47,
  "cached": 0,
  "gen": 31,
  "finish": "stop",
  "think_words": 0,
  "ans_words": 8,
  "prefill_ms": 104.4,
  "prefill_tok_s": 450.2,
  "decode_tok_s": 49.9,
  "decode_estimated": false,
  "tau": null,
  "cycles": null,
  "dflash": null,
  "mtp": null,
  "mtp_ngram": null,
  "ngram_mod_windows": null,
  "ngram_mod_drafts": null,
  "ngram_mod_accepted": null,
  "ngram_mod_accept_rate": null,
  "mtp_windows": null,
  "ar_windows": null,
  "mtp_retired": null,
  "mtp_window_timings": null,
  "ttft_s": 0.125,
  "wall_s": 0.745,
  "attractor": false,
  "empty": false,
  "runaway": false,
  "ans_preview": "{\n  \"name\": \"Alice\",\n  \"age\": 34,\n  \"city\": \"Lisbon\"\n}",
  "assistant_content": "{\n  \"name\": \"Alice\",\n  \"age\": 34,\n  \"city\": \"Lisbon\"\n}",
  "content": "{\n  \"name\": \"Alice\",\n  \"age\": 34,\n  \"city\": \"Lisbon\"\n}",
  "reasoning_content": "",
  "tool_calls": [],
  "request_md5": "9d087beafcbdb8d117c1ae8df956b758",
  "atem_leak": false,
  "prompt_md5": "92bba2040a26fc8018f8e00769bccec9",
  "expected_substrings": [
   "\"name\"",
   "Alice",
   "\"age\"",
   "34",
   "\"city\"",
   "Lisbon"
  ],
  "retrieval_missing": []
 },
 {
  "request_id": "chatcmpl-1085773-9",
  "ctx": 47,
  "cached": 0,
  "gen": 6,
  "finish": "stop",
  "think_words": 0,
  "ans_words": 2,
  "prefill_ms": 104.5,
  "prefill_tok_s": 449.7,
  "decode_tok_s": 50.0,
  "decode_estimated": false,
  "tau": null,
  "cycles": null,
  "dflash": null,
  "mtp": null,
  "mtp_ngram": null,
  "ngram_mod_windows": null,
  "ngram_mod_drafts": null,
  "ngram_mod_accepted": null,
  "ngram_mod_accept_rate": null,
  "mtp_windows": null,
  "ar_windows": null,
  "mtp_retired": null,
  "mtp_window_timings": null,
  "ttft_s": 0.125,
  "wall_s": 0.245,
  "attractor": false,
  "empty": false,
  "runaway": false,
  "ans_preview": "Answer: 43",
  "assistant_content": "Answer: 43",
  "content": "Answer: 43",
  "reasoning_content": "",
  "tool_calls": [],
  "request_md5": "aaa69fa09d3048ef41d1e4642ff61026",
  "atem_leak": false,
  "prompt_md5": "eb63dba2cde2d0c46144775dbf19d7c8",
  "expected_substrings": [
   "Answer: 43"
  ],
  "retrieval_missing": []
 }
]
// --- q36-chain.json
[
 {
  "request_id": "chatcmpl-1085975-1",
  "ctx": 34,
  "cached": 0,
  "gen": 144,
  "finish": "stop",
  "think_words": 0,
  "ans_words": 67,
  "prefill_ms": 114.6,
  "prefill_tok_s": 296.6,
  "decode_tok_s": 49.7,
  "decode_estimated": false,
  "tau": null,
  "cycles": null,
  "dflash": null,
  "mtp": null,
  "mtp_ngram": null,
  "ngram_mod_windows": null,
  "ngram_mod_drafts": null,
  "ngram_mod_accepted": null,
  "ngram_mod_accept_rate": null,
  "mtp_windows": null,
  "ar_windows": null,
  "mtp_retired": null,
  "mtp_window_timings": null,
  "ttft_s": 0.124,
  "wall_s": 3.017,
  "attractor": false,
  "empty": false,
  "runaway": false,
  "ans_preview": "```python\ndef longest_substring_without_repeating(s: str) -> str:\n    if not s:\n        re",
  "assistant_content": "```python\ndef longest_substring_without_repeating(s: str) -> str:\n    if not s:\n        return \"\"\n    \n    start = 0\n    max_start = 0\n    max_length = 0\n    char_index = {}\n    \n    for end, char in enumerate(s):\n        if char in char_index and char_index[char] >= start:\n            start = char_index[char] + 1\n        \n        char_index[char] = end\n        \n        if end - start + 1 > max_length:\n            max_length = end - start + 1\n            max_start = start\n    \n    return s[max_start:max_start + max_length]\n```",
  "content": "```python\ndef longest_substring_without_repeating(s: str) -> str:\n    if not s:\n        return \"\"\n    \n    start = 0\n    max_start = 0\n    max_length = 0\n    char_index = {}\n    \n    for end, char in enumerate(s):\n        if char in char_index and char_index[char] >= start:\n            start = char_index[char] + 1\n        \n        char_index[char] = end\n        \n        if end - start + 1 > max_length:\n            max_length = end - start + 1\n            max_start = start\n    \n    return s[max_start:max_start + max_length]\n```",
  "reasoning_content": "",
  "tool_calls": [],
  "request_md5": "ac7884ced07cabc95cd665752c0ede05",
  "atem_leak": false,
  "prompt_md5": "caf1acdae66df8381a3ce1b0727886ca",
  "expected_substrings": [
   "def "
  ],
  "retrieval_missing": []
 },
 {
  "request_id": "chatcmpl-1085975-3",
  "ctx": 209,
  "cached": 179,
  "gen": 82,
  "finish": "stop",
  "think_words": 0,
  "ans_words": 70,
  "prefill_ms": 89.2,
  "prefill_tok_s": 336.1,
  "decode_tok_s": 49.7,
  "decode_estimated": false,
  "tau": null,
  "cycles": null,
  "dflash": null,
  "mtp": null,
  "mtp_ngram": null,
  "ngram_mod_windows": null,
  "ngram_mod_drafts": null,
  "ngram_mod_accepted": null,
  "ngram_mod_accept_rate": null,
  "mtp_windows": null,
  "ar_windows": null,
  "mtp_retired": null,
  "mtp_window_timings": null,
  "ttft_s": 0.091,
  "wall_s": 1.739,
  "attractor": false,
  "empty": false,
  "runaway": false,
  "ans_preview": "During the day, sunlight passes through a thinner layer of Earth's atmosphere, causing sho",
  "assistant_content": "During the day, sunlight passes through a thinner layer of Earth's atmosphere, causing shorter blue wavelengths to scatter widely in all directions via Rayleigh scattering. At sunset, the sun is lower on the horizon, so light travels through a much thicker layer of atmosphere, scattering away most of the blue light. This leaves predominantly longer red and orange wavelengths to reach our eyes, creating the vibrant hues seen at dusk.",
  "content": "During the day, sunlight passes through a thinner layer of Earth's atmosphere, causing shorter blue wavelengths to scatter widely in all directions via Rayleigh scattering. At sunset, the sun is lower on the horizon, so light travels through a much thicker layer of atmosphere, scattering away most of the blue light. This leaves predominantly longer red and orange wavelengths to reach our eyes, creating the vibrant hues seen at dusk.",
  "reasoning_content": "",
  "tool_calls": [],
  "request_md5": "661bb978a4c81eafee309f97e059fbcf",
  "atem_leak": false,
  "prompt_md5": "1579a134be91ab67653a176a5cb3007d",
  "expected_substrings": [
   "scatter"
  ],
  "retrieval_missing": []
 },
 {
  "request_id": "chatcmpl-1085975-5",
  "ctx": 323,
  "cached": 292,
  "gen": 16,
  "finish": "stop",
  "think_words": 0,
  "ans_words": 13,
  "prefill_ms": 90.3,
  "prefill_tok_s": 343.3,
  "decode_tok_s": 49.7,
  "decode_estimated": false,
  "tau": null,
  "cycles": null,
  "dflash": null,
  "mtp": null,
  "mtp_ngram": null,
  "ngram_mod_windows": null,
  "ngram_mod_drafts": null,
  "ngram_mod_accepted": null,
  "ngram_mod_accept_rate": null,
  "mtp_windows": null,
  "ar_windows": null,
  "mtp_retired": null,
  "mtp_window_timings": null,
  "ttft_s": 0.092,
  "wall_s": 0.414,
  "attractor": false,
  "empty": false,
  "runaway": false,
  "ans_preview": "The capital of France is Paris, and the Seine River runs through it.",
  "assistant_content": "The capital of France is Paris, and the Seine River runs through it.",
  "content": "The capital of France is Paris, and the Seine River runs through it.",
  "reasoning_content": "",
  "tool_calls": [],
  "request_md5": "63fff158a3cf6945aef0536f65001c99",
  "atem_leak": false,
  "prompt_md5": "8caca580290fb75cea2524f52dd6d094",
  "expected_substrings": [
   "Paris",
   "Seine"
  ],
  "retrieval_missing": []
 },
 {
  "request_id": "chatcmpl-1085975-7",
  "ctx": 387,
  "cached": 340,
  "gen": 36,
  "finish": "stop",
  "think_words": 0,
  "ans_words": 10,
  "prefill_ms": 123.8,
  "prefill_tok_s": 379.6,
  "decode_tok_s": 49.7,
  "decode_estimated": false,
  "tau": null,
  "cycles": null,
  "dflash": null,
  "mtp": null,
  "mtp_ngram": null,
  "ngram_mod_windows": null,
  "ngram_mod_drafts": null,
  "ngram_mod_accepted": null,
  "ngram_mod_accept_rate": null,
  "mtp_windows": null,
  "ar_windows": null,
  "mtp_retired": null,
  "mtp_window_timings": null,
  "ttft_s": 0.125,
  "wall_s": 0.85,
  "attractor": false,
  "empty": false,
  "runaway": false,
  "ans_preview": "```json\n{\n  \"name\": \"Alice\",\n  \"age\": 34,\n  \"city\": \"Lisbon\"\n}\n```",
  "assistant_content": "```json\n{\n  \"name\": \"Alice\",\n  \"age\": 34,\n  \"city\": \"Lisbon\"\n}\n```",
  "content": "```json\n{\n  \"name\": \"Alice\",\n  \"age\": 34,\n  \"city\": \"Lisbon\"\n}\n```",
  "reasoning_content": "",
  "tool_calls": [],
  "request_md5": "dd74c83f2745dd73d8ab78c74c1873c5",
  "atem_leak": false,
  "prompt_md5": "92bba2040a26fc8018f8e00769bccec9",
  "expected_substrings": [
   "\"name\"",
   "Alice",
   "\"age\"",
   "34",
   "\"city\"",
   "Lisbon"
  ],
  "retrieval_missing": []
 },
 {
  "request_id": "chatcmpl-1085975-9",
  "ctx": 471,
  "cached": 424,
  "gen": 17,
  "finish": "stop",
  "think_words": 0,
  "ans_words": 7,
  "prefill_ms": 123.4,
  "prefill_tok_s": 380.8,
  "decode_tok_s": 49.7,
  "decode_estimated": false,
  "tau": null,
  "cycles": null,
  "dflash": null,
  "mtp": null,
  "mtp_ngram": null,
  "ngram_mod_windows": null,
  "ngram_mod_drafts": null,
  "ngram_mod_accepted": null,
  "ngram_mod_accept_rate": null,
  "mtp_windows": null,
  "ar_windows": null,
  "mtp_retired": null,
  "mtp_window_timings": null,
  "ttft_s": 0.125,
  "wall_s": 0.468,
  "attractor": false,
  "empty": false,
  "runaway": false,
  "ans_preview": "17 + 26 = 43\n\nAnswer: 43",
  "assistant_content": "17 + 26 = 43\n\nAnswer: 43",
  "content": "17 + 26 = 43\n\nAnswer: 43",
  "reasoning_content": "",
  "tool_calls": [],
  "request_md5": "3e772dd4f9100931dca5c96afbc99376",
  "atem_leak": false,
  "prompt_md5": "eb63dba2cde2d0c46144775dbf19d7c8",
  "expected_substrings": [
   "Answer: 43"
  ],
  "retrieval_missing": []
 }
]

Hardware validation request (optional)

{
  "routes": [
    {"mode": "battery", "tag": "qwen3.6:27b"},
    {"mode": "chain",   "tag": "qwen3.6:27b"}
  ],
  "claim": "tool-free serve on the speculative route is unchanged; a request carrying tools now yields parsed calls on that route regardless of the grammar switch."
}

Related: #220 (tools calling), #646 Track A (tool parity on the slot path).

Architecture-trait change?

No.

…, independent of the grammar switch

Kaden-Schutt · 2026-09-07T04:03:36Z

Superseded by #729, which landed on beta (same enable_grammar split, with a grammar-off regression test). Ships in #730. The emit_text malformed-body log line from here is worth lifting as a follow-up.

FIX(qwen35-spec): parse tool calls whenever the request carries tools…

a46ec79

…, independent of the grammar switch

alpineQ requested a review from Kaden-Schutt as a code owner September 5, 2026 10:12

bhncat mentioned this pull request Sep 6, 2026

fix(qwen35): parse XML tool calls when grammar is off #729

Open

16 tasks

Kaden-Schutt mentioned this pull request Sep 7, 2026

v0.3.1: beta → master — reconciled ladder, DFlash cache repair, admission, oom_guard, hw-gate 3.8 pin #730

Draft

6 tasks

Kaden-Schutt closed this Sep 7, 2026

Navigation Menu

Uh oh!

fix(qwen35-spec): parse tool calls whenever the request carries tools, independent of the grammar switch - #727

fix(qwen35-spec): parse tool calls whenever the request carries tools, independent of the grammar switch#727
alpineQ wants to merge 1 commit into
warpfront:masterfrom
alpineQ:fix/spec-route-tool-parse

alpineQ commented Sep 5, 2026

Uh oh!

Kaden-Schutt commented Sep 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

alpineQ commented Sep 5, 2026

Summary

What was wrong

What changes

Which surface(s) does this touch?

Test plan

Hardware validation request (optional)

Architecture-trait change?

Uh oh!

Kaden-Schutt commented Sep 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants