Skip to content

Normalize structured typed tool results for Qwen chat templates #528

Description

@leehack

Typed LlamaToolResultContent with a JSON object result fails before inference with Exception: Unexpected content type. when rendering the Qwen3.5 template.

Reproduced with Qwen3.5-0.8B Q4_0 (model SHA256 57d1997790d1744fba5b40a7317df71ea5e2acee28c47e78f0cce39c0703f8cf, HF revision 8fea620810c4afa23dd6443f999a48574c1611a3), native v0.4.1, explicit CPU, context1024/threads4/temp0/seed1, on both main 6c01a69af5bd13b338763f5f63cf16e482aa9fd6 and validation baseline 644535c1440a0ae6c48afb5f24ec42fd014283da.

History contains a valid assistant get_weather call for Montréal, typed result {'city':'Montréal','temperature_celsius':17}, and a user asking for the numeric temperature. The initial auto call succeeds; the typed-result followup fails.

LlamaChatMessage.toJson() retains the raw result Map as content. TemplateRenderContext passes it through. The exact model template's render_content macro rejects mapping-valued content. Stack goes through dinja raise_exception, Qwen3CoderXmlHandler.render, ChatTemplateEngine.render, ChatTemplateRenderer.render, then LlamaEngine.create.

A model-free control using the identical template/history fails with the Map but renders successfully with JSON text, preserving the exact JSON payload. This is a Dart template-input issue, not GPU execution or native LiteRT. Current main excludes validation PR #515 and reproduces; no attribution to #515 or the recent stop/cleanup fixes.

Proposed scope: normalize structured typed tool results at the template render boundary while preserving strings, source objects, public serialization and multimodal parts. Retain the original failing suite oracle. Cover exact-template rendering, arrays/scalars/null, escaping/Unicode, source immutability, ordinary media, and branch-bypass regression tests before proposing a fix.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions