fix(qwen35): parse XML tool calls when grammar is off - #729
Open
bhncat wants to merge 1 commit into
Open
Conversation
Qwen3.8 (and other XML-native Qwen3.5/3.6 cards) default HIPFIRE_QWEN35_GRAMMAR off. The DFlash path treated that as "do not pass tools into the emitter," which also disabled ToolOutputRouter. The model still emitted native <tool_call><function=...> XML, but the HTTP API returned it as assistant content with finish_reason=stop and no message.tool_calls. OpenAI clients such as Hermes never executed the tools. Keep grammar off for those cards, but still forward tools so the parser runs. Constrained grammar stays a separate flag (enable_grammar).
Kaden-Schutt
added a commit
that referenced
this pull request
Sep 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stock Qwen3.8 (and other XML-native Qwen3.5/3.6 cards) default
HIPFIRE_QWEN35_GRAMMARoff. The DFlash path treated that as "do not pass tools into the emitter," which also disabledToolOutputRouter. The model still emitted native<tool_call><function=...>XML, but the HTTP API returned it as assistant content withfinish_reason=stopand nomessage.tool_calls. OpenAI clients such as Hermes never executed the tools.Keep grammar off for those cards, but still forward tools so the parser runs. Constrained grammar stays a separate flag (
enable_grammar).Related: #727 (same diagnosis on the speculative route; this branch is an independent gfx1201 DFlash / Hermes confirmation with a unit test for grammar-off XML).
Which surface(s) does this touch?
hw-gate selects hardware routes from the diff (
scripts/hw-gate/select.py); tick what applies so a reviewer can check the selection.kernels/,crates/rdna-compute,crates/hipfire-dispatch,crates/hip-bridge,crates/saddle-corecrates/hipfire-loader,crates/hipfire-daemon, runtime load path (model_load,hfq,loader_api,config,safetensors_source,weight_backend,multi_gpu), archload*/weights*/carrier.rs,hipfire-config,hipfire-registry,registry/, Cargo manifestscrates/hipfire-engine,crates/hipfire-generate, daemon slots/serve, runtime emit/eos/dflash/dspark/spec/reset/triattnhipfire-arch-qwen35crates/hipfire-quantize/ quant formats (updatedocs/quant-formats/qt-register.txt)hipfire-cli,hipfire-client,hipfire-tui.github/workflows/,CODEOWNERS,scripts/hw-gate/,leanup-thresholds.txt,layering.txt,registry/(hard floor: no seat can merge these; a human does)Test plan
./scripts/no-gpu-ci.shpasses, or the CI jobs are greencargo build --releaseclean (-p hipfire-cli -p hipfire-daemon --locked)cargo test --lib --workspacepassesxml_tool_call_held_when_tools_present_and_grammar_offinhipfire-arch-qwen35qwen3.8:27bDFlashget_timeprobe returnedfinish_reason=tool_calls,tool_calls[0].function.name=get_time, empty contentserve_harness.py batterynot attached (happy to run if the gate wants it). Ahipfire runtranscript is not evidence../scripts/speed-gate.shwithin ±2% of locked baselinesscripts/leanup-thresholds.txt: the commit message carriesRATCHET-RAISE: <metric> <old> -> <new>, traded for <reason>and the PR carries theratchet-raiselabel (CI fails without both)No
Architecturetrait change.Hardware validation request (optional)
Tell the gate which registry artifacts prove your change, and what you claim. Sol reads the claim as a claim and runs the routes you name (tags must exist on the runner; unknown or absent tags are reported, not failed). Leave the block out and the gate runs the mandatory fixtures for the surfaces you touched.
{ "routes": [ {"mode": "battery", "tag": "qwen3.8:27b"} ], "claim": "Qwen3.5-family DFlash serve still loads and an XML-native tool call is lifted into message.tool_calls when grammar is off" }