Skip to content

Fix schema-aware Qwen XML tool-call parsing - #531

Merged
leehack merged 1 commit into
mainfrom
fix/qwen-schema-aware-tool-parsing
Sep 19, 2026
Merged

leehack merged 1 commit into
mainfrom
fix/qwen-schema-aware-tool-parsing

Conversation

@leehack

@leehack leehack commented Sep 19, 2026

Copy link
Copy Markdown
Owner

Summary

Qwen XML tool calls now honor supplied tool schemas: a declared string such as "123" stays a string, and undeclared functions, duplicate/unknown parameters, missing required values, or invalid types produce no executable call. Invalid final output rolls back to raw content atomically.

Fixes #530.

Production-readiness scope

Completeness checklist

Test Plan

  • Workspace preparation, exact CI SDK formatting, analysis.
  • Full VM: 2,280 passed, 3 skipped; line coverage 81.11% (70% gate).
  • Full Chrome: 979 passed.
  • Focused VM/Chrome: 55 each; public-engine exact-template tests: 8.
  • Pinned upstream/parity suite: 74 detection, 84 diagnostics, 525 template-unit, 3 upstream E2E, 16 compiled grammar tests passed.
  • Documentation build and link validation.
  • Actual Qwen CPU auto/required calls preserve code as a string; result follow-ups return 123, and recovery passes.
  • Disposable combined Fix structured tool-result rendering for Qwen templates #529 + this fix: all four original strict typed-Map oracles pass unchanged; neither existing branch modified.

Earlier full-run tooling timeouts are preserved in local evidence; the final entire run passed with unchanged timeouts. A first real-model follow-up produced quoted digits; an explicit digits-only prompt passed the unchanged strict output/type assertions.

Matrix Evidence

Scope Platform / model / backend Result Notes
Schema, streaming rollback, choices/thinking VM and Chrome / deterministic emissions PASS Positive, malformed, undeclared, duplicate and required-value cases
Public engine VM / exact Qwen3.5 template / mock backend PASS auto/required/none, thinking, recovery
Pinned upstream and grammar Native compiled upstream b2e5e9b PASS Actual prompt parity and grammar acceptance/rejection; schema types enforced by parser
Model lifecycle macOS / Qwen3.5-0.8B-Q4_0 / CPU PASS context 1024, threads 4, GPU layers 0, temperature 0, seed 1, thinking false
Accelerators / other platforms GPU, NPU, device matrix N/A No new backend or platform capability claim

High-risk regression review

  • Classification: high-risk parser / structured-output change.
  • Implementation task: issue Qwen3.5 XML parser loses schema strings and exposes undeclared tool calls #530 standalone parser fix.
  • Independent blocking QA: codex-adversarial-qwen-schema-0146fda1; exact-commit review accepted with 42 independently passing tests.
  • Exact head / current base: 0146fda15c4e4375cd72b79fe49f209e9a8d891c / 6c01a69af5bd13b338763f5f63cf16e482aa9fd6.
  • Bypass proof: Four isolated mutations caught: missing schema forwarding, declared-name guard, duplicate-key guard, required-value validation.
  • Affected-family evidence: Actual Qwen CPU lifecycle, exact template with pinned upstream emissions, compiled grammar tests. Upstream drops invalid content; Dart intentionally preserves raw content, so invalid-output comparison concerns callable behavior.
  • Six-axis PR-bound evaluator: Independent PR531-bound audit accepted and all six changed-test axes are internally consistent. Evaluator returns unverifiedPrerequisites / externalPrerequisitesUnavailable: local evidence cannot authenticate the dedicated GitHub App, independent auditor, protected environment or conditional ruleset. This is not operational merge readiness; remains draft.
  • Known PR-caused P1 regressions: 0.
  • Unresolved review threads: 0 (independently queried on the exact head).
  • Hosted CI: All 14 checks SUCCESS on exact head 0146fda15c4e4375cd72b79fe49f209e9a8d891c, including Linux coverage, macOS, Windows, Chrome, Web Chat Contract, companions, docs and preview. No fixes or reruns required.

@github-actions

github-actions Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Chat app preview removed for leehack/llamadart-chat-pr-531.

@leehack
leehack marked this pull request as ready for review September 19, 2026 00:57
@leehack
leehack merged commit 04cdaa9 into main Sep 19, 2026
15 checks passed
@leehack
leehack deleted the fix/qwen-schema-aware-tool-parsing branch September 19, 2026 00:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Qwen3.5 XML parser loses schema strings and exposes undeclared tool calls

1 participant