Bump llama.cpp to a646006 (b9932) for v0.8.34

CHANGELOG.md

@@ -1,5 +1,27 @@
 # Changelog
 
+## v0.8.34
+
+Maintenance release: llama.cpp bump to b9932. Full suite against the rebuilt
+NIF: 191 tests, 0 failures.
+
+### Changed
+
+- **llama.cpp submodule** — Updated from cb295bf59 to a646006f0 (44 commits, tag b9932). No NIF changes were required: the header diffs in this range are additive only — new `GGML_TYPE_Q2_0`/`GGML_FTYPE_MOSTLY_Q2_0`/`LLAMA_FTYPE_MOSTLY_Q2_0` enum values in `ggml.h`/`llama.h`, a new `common_speculative_init_result` helper in `common/speculative.h`, and a `server_base` field plus `<fstream>` include in `common/common.h`.
+  - **llama core**: fix allowed decreasing positions in a sequence in llama-batch (#25449); add `n_keep_tail` in `split_equal` for recurrent models (#25278); refactor fused ops (#24646); fix quantized KV cache for dsv4 (#25202); fix OOB reads in the UGM tokenizer's `precompiled_charsmap` handling (#18750).
+  - **speculative**: fix out-of-bounds read in ngram-map on prompt shrink (#23936); fix draft-model fit vs load inconsistency in the server (#25056); naming/spacing cleanup (#25410).
+  - **ggml**: add Q2_0 quantization type definition + CPU backend (#24448); CPU f16→f16 `GGML_OP_SET_ROWS` (#25344); fix A-indexing in the simd_gemm scalar tail-column path (#25390); make `ggml_time_init` idempotent (#24422); better default thread count on ppc/AIX (#25237).
+  - **Metal**: add `set_rows` with f16 src0 (#25434); add `col2im_1d` op for f32/f16/bf16 (#25176).
+  - **CUDA**: f16→f16 `SET_ROWS` (#25367); fuse MMVQ post-scale for NVFP4 (#24481); remove `-sm row`, refactor cuBLAS (#24216).
+  - **Vulkan**: disable FA `mask_opt` on GCN (#24362); reduce submission threshold on small AMD GPUs by CU count (#25240); guard unimplemented f16 `SET_ROWS` (#25351).
+  - **OpenCL**: ragged-tile MoE prefill FP16 GEMM optimization (#25433); flash-attention decode perf (#25366); fix potential crash in aos reconstruct (#25383).
+  - **SYCL/HIP/hexagon**: eight SYCL commits (col2im_1d, cross-entropy-loss ops, argsort coverage, AOT double fix, env-var renames); HIP `-fno-finite-math-only` alongside `-ffast-math` (#25373); hexagon VTCM layouts + pipeline improvements for MUL_MAT/MUL_MAT_ID/FLASH_ATTN_EXT (#25425).
+  - **tools/server/ui** (not linked into the binding): llama-cli moved to an HTTP-based implementation (#24948); SSE replay-buffer follow-up (#25047); timings/progress in `/responses` API streams (#25348); prompt-cache RAM limit enforcement (#25070); fix `load_models()` deadlock (#25358); context-usage gauge in the web UI (#25340).
+
+### Added
+
+- **Speculative-decoding docs** — README and `LlamaCppEx.MTP` moduledoc now state which upstream speculative types the binding exposes (MTP only) and document DFlash status on Apple Silicon: functional end-to-end on Metal at b9932 via upstream tools, but measured slower than plain decoding at small target sizes (Qwen3.5-4B + 0.6B drafter on M4 Max: 42 tok/s vs 85 tok/s plain, 30% acceptance greedy), with community drafter-GGUF conversions still incompatible across converter revisions (#25116, #25110).
+
 ## v0.8.33
 
 Performance and robustness release for the batching `Server` and generation paths

README.md

@@ -494,6 +494,12 @@ Multi-Token Prediction speculative decoding (upstream PR [#22673](https://github
 >
 > Larger `n_draft` hurts on Metal because verify cost grows faster than acceptance benefit. On NVIDIA, `n_draft: 3` is the right default — that's what the upstream 2× number assumes.
 
+### Other speculative types (EAGLE-3, DFlash, n-gram)
+
+Upstream llama.cpp implements more speculative types behind the same `common_speculative` API — `draft-eagle3`, `draft-dflash` (block-diffusion drafting via a separate drafter GGUF), and several n-gram self-speculation modes. **This binding currently exposes only MTP**: `MTP.init/2` pins `COMMON_SPECULATIVE_TYPE_DRAFT_MTP` and builds both contexts from the same model, so there is no way to load a separate drafter model yet.
+
+> **DFlash status (July 2026, llama.cpp b9932).** DFlash runs end-to-end on Metal via upstream `llama-cli`/`llama-server`, but we measured it *slower* than plain decoding on Apple Silicon at small target sizes: Qwen3.5-4B target + z-lab 0.6B drafter on M4 Max reached 42 tok/s with DFlash vs 85 tok/s plain (greedy sampling; 30% draft acceptance, mean accepted run 2.8 — and stochastic sampling at `temp 0.8` collapses acceptance to ~7%). The Metal economics are the same as the MTP note above (wide verify batches are expensive), and the community drafter-GGUF conversions are still churning: of three third-party Qwen 4B drafter repos tested, only one loads with current upstream (the others hit the `dflash-draft` arch mismatch [#25116](https://github.com/ggml-org/llama.cpp/issues/25116) or lack the `target_layers` metadata added by the conversion refactor [#25110](https://github.com/ggml-org/llama.cpp/pull/25110)). Worth revisiting when the drafter format settles; the natural entry point is a `spec_type` + drafter-model option on `speculative_init`.
+
 ### Models with MTP heads
 
 - [`ggml-org/Qwen3.6-35B-A3B-MTP-GGUF`](https://huggingface.co/ggml-org/Qwen3.6-35B-A3B-MTP-GGUF) (recommended: `Q4_K_M`, ~21 GB)

lib/llama_cpp_ex/mtp.ex

@@ -30,6 +30,14 @@ defmodule LlamaCppEx.MTP do
   that — a single MTP session decodes one sequence at a time. Reuse the same
   `%MTP{}` value across calls to `stream/3` / `generate/3` to avoid rebuilding
   the contexts; KV caches are cleared on each call.
+
+  MTP is the only speculative type this binding exposes. Upstream llama.cpp
+  also implements EAGLE-3, DFlash (block-diffusion drafting via a separate
+  drafter GGUF), and n-gram self-speculation behind the same
+  `common_speculative` API, but the NIF pins the MTP type and both contexts
+  are built from the same model, so a separate drafter model cannot be loaded
+  yet. See the "Speculative decoding" section of the README for the current
+  status of DFlash on Apple Silicon.
   """
 
   alias LlamaCppEx.{Context, Model, Sampler, Tokenizer}

mix.exs

@@ -37,7 +37,7 @@ end
 defmodule LlamaCppEx.MixProject do
   use Mix.Project
 
-  @version "0.8.33"
+  @version "0.8.34"
   @source_url "https://github.com/nyo16/llama_cpp_ex"
 
   def project do