Skip to content

fix(qwen35): parse XML tool calls when grammar is off - #729

Open
bhncat wants to merge 1 commit into
warpfront:masterfrom
bhncat:fix/qwen-xml-tool-calls
Open

fix(qwen35): parse XML tool calls when grammar is off#729
bhncat wants to merge 1 commit into
warpfront:masterfrom
bhncat:fix/qwen-xml-tool-calls

Conversation

@bhncat

@bhncat bhncat commented Sep 6, 2026

Copy link
Copy Markdown

Summary

Stock Qwen3.8 (and other XML-native Qwen3.5/3.6 cards) default HIPFIRE_QWEN35_GRAMMAR off. The DFlash path treated that as "do not pass tools into the emitter," which also disabled ToolOutputRouter. The model still emitted native <tool_call><function=...> XML, but the HTTP API returned it as assistant content with finish_reason=stop and no message.tool_calls. OpenAI clients such as Hermes never executed the tools.

Keep grammar off for those cards, but still forward tools so the parser runs. Constrained grammar stays a separate flag (enable_grammar).

Related: #727 (same diagnosis on the speculative route; this branch is an independent gfx1201 DFlash / Hermes confirmation with a unit test for grammar-off XML).

Which surface(s) does this touch?

hw-gate selects hardware routes from the diff (scripts/hw-gate/select.py); tick what applies so a reviewer can check the selection.

  • kernelkernels/, crates/rdna-compute, crates/hipfire-dispatch, crates/hip-bridge, crates/saddle-core
  • loadcrates/hipfire-loader, crates/hipfire-daemon, runtime load path (model_load, hfq, loader_api, config, safetensors_source, weight_backend, multi_gpu), arch load*/weights*/carrier.rs, hipfire-config, hipfire-registry, registry/, Cargo manifests
  • servecrates/hipfire-engine, crates/hipfire-generate, daemon slots/serve, runtime emit/eos/dflash/dspark/spec/reset/triattn
  • arch crate(s): hipfire-arch-qwen35
  • crates/hipfire-quantize / quant formats (update docs/quant-formats/qt-register.txt)
  • control plane — hipfire-cli, hipfire-client, hipfire-tui
  • docs / CI / scripts only (no hardware route)
  • policy files.github/workflows/, CODEOWNERS, scripts/hw-gate/, leanup-thresholds.txt, layering.txt, registry/ (hard floor: no seat can merge these; a human does)

Test plan

  • ./scripts/no-gpu-ci.sh passes, or the CI jobs are green
  • cargo build --release clean (-p hipfire-cli -p hipfire-daemon --locked)
  • cargo test --lib --workspace passes
  • Unit: xml_tool_call_held_when_tools_present_and_grammar_off in hipfire-arch-qwen35
  • Live gfx1201: qwen3.8:27b DFlash get_time probe returned finish_reason=tool_calls, tool_calls[0].function.name=get_time, empty content
  • load / serve / kernel changes: serve_harness.py battery not attached (happy to run if the gate wants it). A hipfire run transcript is not evidence.
  • If perf-relevant: ./scripts/speed-gate.sh within ±2% of locked baselines
  • If this raises a ceiling in scripts/leanup-thresholds.txt: the commit message carries RATCHET-RAISE: <metric> <old> -> <new>, traded for <reason> and the PR carries the ratchet-raise label (CI fails without both)

No Architecture trait change.

Hardware validation request (optional)

Tell the gate which registry artifacts prove your change, and what you claim. Sol reads the claim as a claim and runs the routes you name (tags must exist on the runner; unknown or absent tags are reported, not failed). Leave the block out and the gate runs the mandatory fixtures for the surfaces you touched.

{
  "routes": [
    {"mode": "battery", "tag": "qwen3.8:27b"}
  ],
  "claim": "Qwen3.5-family DFlash serve still loads and an XML-native tool call is lifted into message.tool_calls when grammar is off"
}

Qwen3.8 (and other XML-native Qwen3.5/3.6 cards) default HIPFIRE_QWEN35_GRAMMAR
off. The DFlash path treated that as "do not pass tools into the emitter,"
which also disabled ToolOutputRouter. The model still emitted native
<tool_call><function=...> XML, but the HTTP API returned it as assistant
content with finish_reason=stop and no message.tool_calls. OpenAI clients
such as Hermes never executed the tools.

Keep grammar off for those cards, but still forward tools so the parser
runs. Constrained grammar stays a separate flag (enable_grammar).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant