Skip to content

GGUF public generation emits caller stop markers before stopping #520

Description

@leehack

The public llama.cpp/GGUF generation path emits a caller-supplied stop marker before ending the stream. The portable catalog-3 C10.stop test reproduces this with both Gemma 4 E2B and Qwen 3.5 0.8B on macOS CPU/Metal and Linux CUDA.

Reproduction: source 89097b704fbfe90718484cd577076e7a5cb7083d, native v0.4.1, deterministic profile settings; public chat prompt Reply with exactly: alpha cedar17 omega, max tokens 32, then the same request with GenerationParams(stopSequences: ['cedar17']).

Gemma control: alpha cedar17 omega. Stopped output: alpha cedar17. Expected visible output: alpha , excluding the stop marker, consistent with the existing LiteRT-LM and Web stop suppression contracts. Subsequent generation succeeds. The unrestricted control proves the model emitted the expected fixture, so this failure is not missing model compliance.

The validation PR adds detection; it does not change core inference. Static inspection finds llama.cpp service loops yielding token bytes before checking text.endsWith(stop) (including lib/src/backends/llama_cpp/llama_cpp_service.dart around 5400), which explains the exposed marker. The fix should cover incremental/partial markers, marker in the middle of a token, Unicode, empty/overlapping stop strings, preserved template tokens, all relevant native generation branches and recovery without losing non-marker output. Do not relax the validation oracle.

Evidence: #515 catalog-3 release matrix, local followup-20260918/macos-release-gemma4-gguf-{cpu,metal}/results.json and Linux L4 CUDA results. CUDA/Metal offload was independently verified for all five model loads. Gemma LiteRT-LM CPU passes C10.stop with the same fixture. Related coverage tracker: #514.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions