fix: suppress caller stop markers in native GGUF streams

[leehack]

Summary

Fixes #520. Native GGUF streams previously yielded a caller stop marker before checking a 64-byte suffix. The public alpha cedar17 omega fixture with stop cedar17 now emits exactly alpha .

The ordinary, speculative sampled, and speculative accepted-token paths share an incremental byte matcher before yielding. It suppresses within-token tails, handles split UTF-8 and overlapping/long markers, and releases unfinished prefixes at completion without losing bytes. Per-marker prefix state bounds matching work; exact preserved template-token exclusions remain intact.

Production-readiness scope

• User-facing scope: Caller stop suppression in native GGUF output, including speculative decoding, with cancellation and next-request recovery.
• Supported platforms/paths: Shared native llama.cpp service; directly verified macOS arm64 CPU/Metal. Web/LiteRT behavior and native artifact pins are unchanged.
• Unsupported paths: Speculative grammar sampling remains explicitly rejected. Tests use deterministic unrestricted speculative controls and require actual draft acceptance.
• Out of scope / follow-ups: New native artifacts, platform expansion, and paid cloud runs. Existing broader platform qualification remains tracked in Complete cross-platform validation cases, device coverage and reports #514; no new feature follow-up.

Completeness checklist

• Declared scope fully implemented.
• Unsupported option combinations retain explicit errors.
• API/runtime docs and unreleased changelog updated; examples/support claims unchanged.
• Positive and negative regression coverage added.
• No new production diagnostics or secret-bearing metadata.
• Broader platform qualification tracked in Complete cross-platform validation cases, device coverage and reports #514.

Test Plan

Flutter 3.47.1 / Dart 3.13.1, exact local head 7a3531e1f88d484ac33d363633a782d5d48ea007.

• Workspace preparation, format, and analysis.
• Full VM: 2,142 passed, 77 skipped.
• Full Chrome: 945 passed.
• All 14 hosted checks passed: CI run, native CPU regression cases verified in Linux/macOS/Windows logs.
• Docs build and broken-link validation.
• Native integration: 9 tests; all five individual emission/flush mutants fail exact-output assertions.
• Public Gemma 4 E2B and Qwen3.5 0.8B on CPU/Metal: control, exact stop suppression under 1/1 and 8/512 batching, and subsequent recovery.

Matrix Evidence

Matrix row	Platform / scope	Result	Evidence
static-format-analyze	Pinned SDK, repository	PASS	604 files unchanged; clean analysis
root-vm	macOS arm64	PASS	2,142 pass / 77 skip
root-chrome	Chrome	PASS	945 pass
coverage-lib	lib, final-head VM	PASS	Local 80.47%, hosted Linux 81.49% vs 70% gate; new matcher 100%
docs-site	Docusaurus	PASS	Build and strict link validation
gguf-stop-sequences	Gemma 4 E2B + Qwen3.5, CPU/Metal	PASS	Exact alpha ; 0 CPU offload, Metal 36/36 and 25/25 offload plus compute buffers
high-risk-exact-head-independent-qa	Fresh independent Codex audit	PASS	Exact head/base accepted; 14 buffer tests independently rerun

Runtime llamadart-native v0.4.1, upstream b29c606e28a01b1bc8c1351026a0fa6e616bf6c4; macOS archive SHA256 41d0a5298f377641484eb8ddf07cf18ed9f8eb7cbd1e0bf7e4cdc7bd5b0b1487 verified against GitHub release digest. Gemma model SHA256 0a2fac16f388b4839f075dedb681357aec3e73a96bd66b413e462b6853550c99; Qwen SHA256 57d1997790d1744fba5b40a7317df71ea5e2acee28c47e78f0cce39c0703f8cf.

Review Notes

Fresh independent audit accepted with zero blocking findings. All 14 hosted checks passed on the exact head below, including Linux/macOS/Windows native suites and Chrome. GraphQL review-thread query currently returns zero unresolved threads. The PR-bound evaluator found internally consistent repository evidence and returned the expected unverifiedPrerequisites (exit 2) because authenticated auditor/App/environment/ruleset enforcement is intentionally unconfigured. Per doc/high_risk_pre_merge_readiness.md, that external control is a separate governance scope, not an additional merge prerequisite; manual independent review remains the repository readiness path. This result does not claim operational enforcement.

High-risk regression review

• Classification: high-risk (backendRuntime, regressionPolicy).
• Implementation task: Issue GGUF public generation emits caller stop markers before stopping #520 fix task.
• Independent blocking QA task: fresh-codex-independent_stop_audit, accepted.
• Exact head / current base: 7a3531e1f88d484ac33d363633a782d5d48ea007 / d819bfc1a8ab784a49c97181df8e5293e8ad7516.
• Production-branch deletion/bypass proof: Each of three emission filters and two EOF flushes was individually bypassed; all five mutants fail regression assertions; restored control passes.
• Affected-family evidence: Gemma 4 and Qwen3.5 public CPU/Metal fixture, verified offload; tiny GGUF speculative acceptance and recovery.
• Unavailable evidence: Windows/Linux Gemma/Qwen CUDA/Vulkan model runs were not performed; no paid cloud use. Hosted native CPU regression suites passed. PR-bound evaluator result is recorded below.
• Known change-caused P1 regressions: 0.
• Unresolved review threads: 0 (requeried from GitHub GraphQL after all hosted checks passed).

PR-bound high-risk evidence and local evaluation

{
  "schema": "llamadart.high-risk-readiness-evidence",
  "schema_version": "1.0.0",
  "timestamp": "2026-09-18T15:46:57.629877Z",
  "correlation_id": "pr526-stop-marker-7a3531e1",
  "repository": "leehack/llamadart",
  "pr_number": 526,
  "expected_pr_head_sha": "7a3531e1f88d484ac33d363633a782d5d48ea007",
  "current_base_sha": "d819bfc1a8ab784a49c97181df8e5293e8ad7516",
  "pr_author": "leehack",
  "classification": "high-risk",
  "surfaces": [
    "backendRuntime",
    "regressionPolicy"
  ],
  "required_matrix_row_ids": [
    "high-risk-exact-head-independent-qa"
  ],
  "matrix_row_evidence": {
    "high-risk-exact-head-independent-qa": {
      "row_id": "high-risk-exact-head-independent-qa",
      "result": "pass",
      "command": "Independent git diff review against exact base; dart test -p vm test/unit/backends/llama_cpp/stop_sequence_buffer_test.dart; final-head native control and five individual production branch mutations",
      "evidence_notes": "Fresh independent audit accepted exact head/base, 14 independent buffer tests pass; parent final-head VM2142/Chrome945, four Gemma4/Qwen3.5 CPU/Metal public smokes, nine native integration tests, and five assertion-failing emission/flush mutants. No known P1 regressions. Windows/Linux GPU real-model smokes not rerun. Hosted CI: all 14 checks pass, including actual new native regression tests on Linux, Windows and macOS. Final GraphQL query has zero review threads."
    }
  },
  "independent_audit": {
    "auditor_identity": "fresh-codex-independent_stop_audit",
    "audit_kind": "codex-adversarial",
    "audit_head_sha": "7a3531e1f88d484ac33d363633a782d5d48ea007",
    "audit_base_sha": "d819bfc1a8ab784a49c97181df8e5293e8ad7516",
    "decision": "accepted",
    "unresolved_review_threads": 0,
    "known_pr_caused_p1_regressions": 0,
    "summary": "No blocking findings. Rechecked all three native emission sites, both completion flushes, preserved-token exclusions, cleanup and recovery; KMP fallback, overlapping matches, split bytes, earliest matching start within each piece and incomplete-prefix flush. Amortized linear incoming-byte matching per marker; pending output bounded by longest marker prefix plus incoming piece."
  },
  "structured_output_evidence": null,
  "affected_test_paths": [
    "test/unit/backends/llama_cpp/stop_sequence_buffer_test.dart",
    "test/unit/backends/llama_cpp/llama_cpp_service_test.dart",
    "test/integration/stop_sequences_test.dart",
    "test/e2e/backends/gguf_stop_sequences_e2e_test.dart"
  ],
  "evaluation": {
    "evaluated_at": "2026-09-18T15:46:58.178626Z",
    "changed_files": [
      {
        "path": "CHANGELOG.md",
        "status": "modified"
      },
      {
        "path": "doc/testing_matrix.md",
        "status": "modified"
      },
      {
        "path": "lib/src/backends/llama_cpp/llama_cpp_service.dart",
        "status": "modified"
      },
      {
        "path": "lib/src/backends/llama_cpp/stop_sequence_buffer.dart",
        "status": "added"
      },
      {
        "path": "lib/src/core/models/inference/generation_params.dart",
        "status": "modified"
      },
      {
        "path": "test/e2e/backends/gguf_stop_sequences_e2e_test.dart",
        "status": "added"
      },
      {
        "path": "test/integration/stop_sequences_test.dart",
        "status": "added"
      },
      {
        "path": "test/unit/backends/llama_cpp/llama_cpp_service_test.dart",
        "status": "modified"
      },
      {
        "path": "test/unit/backends/llama_cpp/stop_sequence_buffer_test.dart",
        "status": "added"
      },
      {
        "path": "tool/testing/run_local_e2e.dart",
        "status": "modified"
      },
      {
        "path": "tool/testing/test_matrix.dart",
        "status": "modified"
      },
      {
        "path": "website/docs/changelog/recent-releases.md",
        "status": "modified"
      },
      {
        "path": "website/docs/configuration/runtime-parameters.md",
        "status": "modified"
      }
    ],
    "decision": "unverifiedPrerequisites",
    "failure_classification": "externalPrerequisitesUnavailable",
    "message": "Repository-local evidence is internally consistent, but auditor authentication, GitHub App publication, protected-environment provenance, and ruleset enforcement are not available. This is not operational merge readiness.",
    "external_prerequisites": {
      "app_installed": false,
      "protected_environment_configured": false,
      "independent_auditor_authenticated": false,
      "ruleset_enforced": false,
      "diagnostic_message": "No repository-local input can authenticate the dedicated GitHub App, protected environment, independent auditor, or conditional ruleset. See doc/high_risk_pre_merge_readiness.md."
    }
  }
}

[github-actions]

Chat app preview removed for leehack/llamadart-chat-pr-526.