Reproduction
Model: DeepSeek-R1-Distill-Qwen-1.5B-Q4_K_M.gguf (qwen2 arch, official bartowski GGUF)
Prompt: any trivial chat prompt (e.g. "What is the capital of France?") with any template (hand or embedded jinja).
- llama.rn 0.12.7 and 0.12.8 (llama.cpp b10156), iOS simulator, CPU-only (gpu_layers: 0, gpuSupported=false): output is degenerate multilingual gibberish from the first token (mixed English/中文/symbols), on every prompt, both prompt paths.
- llama.cpp b10150 standalone llama-server on macOS (Apple Silicon), forced CPU (-ngl 0): same GGUF, same prompt → coherent English output.
- Same GGUF also clean via node-llama-cpp (Metal).
Why this looks like a build-config issue, not an upstream version issue
b10156 ≈ b10150 (same generation), yet the standalone Mac CPU build is fine and llama.rn's iOS simulator build is not. All other archs we tested on the same llama.rn CPU path (qwen3, qwen3.5, gemma, smollm, LFM) are fine — only qwen2 breaks. Points at a kernel/flag difference in the iOS xcframework's CPU slice for qwen2 (e.g. SIMD/accelerate flags), or possibly the deliberate GPU-off-on-simulator path interacting with it.
Happy to provide: the exact build config you want compared (GGML_* flags), the raw first-token output, or to test a patch build. Evidence trail: https://github.com/KA-devl/kernelai/pull/68
Reproduction
Model: DeepSeek-R1-Distill-Qwen-1.5B-Q4_K_M.gguf (qwen2 arch, official bartowski GGUF)
Prompt: any trivial chat prompt (e.g. "What is the capital of France?") with any template (hand or embedded jinja).
Why this looks like a build-config issue, not an upstream version issue
b10156 ≈ b10150 (same generation), yet the standalone Mac CPU build is fine and llama.rn's iOS simulator build is not. All other archs we tested on the same llama.rn CPU path (qwen3, qwen3.5, gemma, smollm, LFM) are fine — only qwen2 breaks. Points at a kernel/flag difference in the iOS xcframework's CPU slice for qwen2 (e.g. SIMD/accelerate flags), or possibly the deliberate GPU-off-on-simulator path interacting with it.
Happy to provide: the exact build config you want compared (GGML_* flags), the raw first-token output, or to test a patch build. Evidence trail: https://github.com/KA-devl/kernelai/pull/68