Fix encoded audio transport in string chat templates

[leehack]

Summary

Fixes #517. Qwen3-ASR's string-only template was receiving serialized audio transport objects: the 352,078-byte encoded JFK WAV became 344,024 text tokens, while the file variant had only 48. Replace audio/image parts with ordered media placeholders only in the string-template rendering copy. Original media stays in the backend request; encoded bytes still use the native buffer decoder.

The same file and encoded bytes now render identical 18-token prompts and produce the exact transcript:

And so, my fellow Americans, ask not what your country can do for you; ask what you can do for your country.

Every reference word is preserved. Correct rendering changes one comma to a semicolon compared with the formerly malformed file prompt. Tests still require full exact equality. Typed templates retain their model-specific wrappers; no API, native loader, artifact pin, or speech sampler changes.

Validation

• Flutter 3.47.1 / Dart 3.13.1: clean workspace preparation, format and analysis.
• Refreshed-head local VM: 642 template/speech/loader tests passed; 15 compiled grammar tests passed. All13 hosted checks passed on the refreshed head, including full Linux VM, macOS/Windows native, Chrome and Web Chat Contract; Linux line coverage79.88% (13408/16785).
• Registered speech-to-text-smoke, macOS arm64 CPU, native v0.4.1, Qwen3-ASR-0.6B Q8_0 + matching projector: file/bytes exact transcript, bounded prompt, cancellation/recovery, malformed-byte rejection/recovery PASS.
• Model/projector/WAV hashes match issue Native Qwen3-ASR byte input fails while the same WAV file transcribes successfully #517. Original-main reproduction: file PASS / bytes FAIL.
• Matching raw native file/buffer controls PASS. Separately built upstream b29c606e28a01b1bc8c1351026a0fa6e616bf6c4 CPU llama-mtmd-cli with identical prompt/model/WAV/context/sampler and 512-token limit returns the exact same transcript.
• Refreshed pinned run_template_parity_suites.sh: 74 detection cases, 78 diagnostic cases, 491 template unit tests, both upstream chat suites, and 15 compiled grammar acceptance/rejection tests PASS.
• Docs build and link checks PASS.
• Independent local exact-head audit codex-adversarial-issue517-audio: no blocking findings, zero known change-caused P1s; 31 focused VM tests and 15 compiled grammar tests independently pass on final head.

Scope and remaining readiness

Real-model evidence is macOS CPU only. No Metal/GPU, mobile, Windows/Linux inference, physical microphone, or real Web model qualification is claimed. Chrome verifies shared template logic. PR #515 was inspected read-only; none of its unmerged validation package was imported.

Reviewed head: 5fcb8bca8bc198e58ba8955028e6d417f525365d; base: 57f9c72996ce789b48336b92755912aaeea704d2. Independent auditor verified actual PR #523 author/head/base. Zero known PR-caused P1 regressions and zero unresolved review threads. Normal merge includes #495 and #522 from current main. All13 refreshed-head hosted checks passed (CI run35348233285). PR remains draft for the coordinating task to perform the approved central merge. The repository evaluator reports internally consistent evidence with the documented unverifiedPrerequisites result (exit 2); authenticated external enforcement is not configured, so this is not an operational-ready attestation. No merge or publication is authorized.

Added scoped audio-plus-structured-output regression coverage for schema reconstruction, split tool-envelope suppression, malformed-output rollback, tool-choice/thinking preservation and compiled grammar acceptance/rejection. These guards use a synthetic MiniMax routing fixture, not model-backed MiniMax audio support. Arbitrary fragmentation of thinking tags is outside the new scoped coverage; the separate synthetic finding is tracked in #524 and is not folded into this PR.

Exact-head high-risk readiness evidence

{
  "schema": "llamadart.high-risk-readiness-evidence",
  "schema_version": "1.0.0",
  "timestamp": "2026-09-18T13:19:08Z",
  "correlation_id": "pr523-5fcb8bca8-base57f9-audio-review",
  "repository": "leehack/llamadart",
  "pr_number": 523,
  "expected_pr_head_sha": "5fcb8bca8bc198e58ba8955028e6d417f525365d",
  "current_base_sha": "57f9c72996ce789b48336b92755912aaeea704d2",
  "pr_author": "leehack",
  "classification": "high-risk",
  "surfaces": [
    "regressionPolicy",
    "structuredOutput"
  ],
  "required_matrix_row_ids": [
    "high-risk-exact-head-independent-qa",
    "structured-output-adversarial"
  ],
  "matrix_row_evidence": {
    "high-risk-exact-head-independent-qa": {
      "row_id": "high-risk-exact-head-independent-qa",
      "result": "pass",
      "command": "Fresh independent exact-head production-call-site review; 31 focused VM tests; 15 compiled grammar tests; focused analysis and git diff --check.",
      "evidence_notes": "Exact refreshed head/base independently verified through Git and GitHub; zero threads/no blockers/zero known PR-caused P1s. Independent31 focusedVM+15compiled PASS. Author reran642 template/speech/loader VM tests,15compiled,publicCPU file/bytes/cancellation/malformed recovery,prepare/format/analyze/docs; allPASS. All13 new-head hosted checksPASS, CI35348233285; Linux line coverage79.88% (13408/16785). Final actual PR head/base/author and zero review threads reverified."
    },
    "structured-output-adversarial": {
      "row_id": "structured-output-adversarial",
      "result": "pass",
      "command": "dart test test/unit/core/template/audio_transport_test.dart; LLAMA_CPP_GBNF_VALIDATOR=.dart_tool/llama_cpp_chat_tests/bin/test-gbnf-validator dart test -p vm -j 1 test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart --run-skipped; tool/testing/run_template_parity_suites.sh",
      "evidence_notes": "Audio-bearing string templates feed actual render results into stream parsing and compiled grammar validation; exact bounded prompts checked first. Wrong names, missing/wrong/extra fields and incomplete envelopes rejected. Scalar/container reconstruction, split envelopes, malformed rollback and auto/required/none with thinking tested. Thinking-prefix token is intact; arbitrary closing-tag fragmentation is not claimed. Qwen ASR real CPU file/bytes and exact-upstream controls match; MiniMax is synthetic pipeline coverage only."
    }
  },
  "independent_audit": {
    "auditor_identity": "codex-adversarial-issue517-audio",
    "audit_kind": "codex-adversarial",
    "audit_head_sha": "5fcb8bca8bc198e58ba8955028e6d417f525365d",
    "audit_base_sha": "57f9c72996ce789b48336b92755912aaeea704d2",
    "decision": "accepted",
    "unresolved_review_threads": 0,
    "known_pr_caused_p1_regressions": 0,
    "summary": "Fresh independent audit accepted merged head against live actual main57f9. Independently verified PR523 head/base/author/draft and zero review threads. Both changelogs preserve audio and loader entries; audio production/tests/runner blobs unchanged from previous audit; incoming thinking-budget loader diagnostics do not change mtmd decoding. Independent31 focusedVM +15compiled tests PASS; diff clean; zero known PR-causedP1. Qwen is actual model evidence; MiniMax is synthetic pipeline only."
  },
  "structured_output_evidence": {
    "coverage": {
      "compiled_grammar_acceptance": [
        "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart"
      ],
      "compiled_grammar_rejection": [
        "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart"
      ],
      "schema_reconstruction": [
        "test/unit/core/template/audio_transport_test.dart"
      ],
      "streaming_rollback": [
        "test/unit/core/template/audio_transport_test.dart"
      ],
      "tool_choice_thinking": [
        "test/unit/core/template/audio_transport_test.dart"
      ],
      "upstream_parity": [
        "test/unit/core/template/audio_transport_test.dart",
        "test/e2e/backends/speech_to_text_e2e_test.dart"
      ]
    },
    "families": [
      {
        "family": "Qwen3-ASR",
        "status": "tested",
        "evidence_test_paths": [
          "test/unit/core/template/audio_transport_test.dart",
          "test/e2e/backends/speech_to_text_e2e_test.dart"
        ],
        "rationale": "Pinned issue517 0.6B-Q8_0 model/projector and JFK WAV hashes verified. macOS CPU v0.4.1 public file/bytes exact transcripts, raw native file/buffer and separately built exact-upstream b29c606 controls agree. Fixed prompt18 tokens versus baseline bytes344024."
      },
      {
        "family": "MiniMax-M3",
        "status": "unavailable",
        "evidence_test_paths": [
          "test/unit/core/template/audio_transport_test.dart",
          "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart"
        ],
        "rationale": "No exact MiniMax weights used. Synthetic string-template routing and primary protocol/schema fixtures verify shared pipeline nonregression only; no real-model MiniMax audio capability claimed."
      }
    ]
  },
  "affected_test_paths": [
    "test/unit/core/template/audio_transport_test.dart",
    "test/e2e/template/specialized_tool_grammar_validation_e2e_test.dart",
    "test/e2e/backends/speech_to_text_e2e_test.dart",
    "test/unit/core/speech/speech_to_text_test.dart"
  ],
  "evaluation": null
}

[github-actions]

Chat app preview removed for leehack/llamadart-chat-pr-523.