Environment: Strix Halo (Ryzen AI MAX+ 395 / gfx1151, 128 GB, 96 GB carve), Windows 11,
TheRock ROCm 10.1.0 SDK, Clang 23, stock runtime (no retained-PM4), model
unsloth Qwen3.8-Flash-Next UD-IQ4_XS (not the PROJFIX quant), -b/-ub 8192,
--load-mode none (lazy-direct unavailable on Windows; PLE on the mmap fallback).
1. Build fix needed (one line)
src/models/qwen4exp.cpp:1776 calls ple_reader->prefetch(...) unconditionally, but the
_WIN32 branch of llama_lazy_reader (llama-lazy-reader.h) only stubs gather().
Fix: add a no-op void prefetch(const int32_t *, int64_t) const {} beside the stubbed
gather (prefetch is a page-cache hint; no-op is behavior-correct on the mmap fallback).
2. Results with that fix (all correctness-gated: math probe + 24k needle, verbatim answers)
- All 35 launcher env gates ON: corrupted output — reasoning degenerates into a slash
flood (//////...), empty content. pp meanwhile 1080–1256 t/s (fast-but-wrong).
- All gates OFF: clean, pp ~418.
- Each family alone (MMB / HC / NORMCONV / QSA): clean.
- Full set minus MMB: clean. Minus HC: clean. Minus QSA: still floods. Minus NORMCONV: no
flood (one ambiguous non-flood wrong-answer run, unreproduced).
- Full set with only
LLAMA_MMB_HC16=0: fully clean — correct at 2k/8k/16k depth,
needle exact at 24k — at pp 964–1045 t/s (262144 ctx), ~6x stock-mainline-class
prefill on this hardware and ~85-90% of the Linux reference numbers, with no custom
runtime and no lazy-direct.
Conclusion: LLAMA_MMB_HC16 (bf16 HC streams end-to-end) miscompiles or misbehaves in
combination with the HC family on Windows/TheRock 10.1/Clang 23. Everything else ports.
Given its ablation worth is only 1.08x, defaulting it off on _WIN32 (or gating it on the
tested runtime) would make the branch Windows-viable as-is.
3. MTP on Windows (informational)
unsloth self-contained mtp-...-Q8_0.gguf loads and drafts via
--spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75: acceptance 77–79%,
but net decode only ~24 t/s (vs ~34–38 on a branch carrying device-resident speculative
checkpoints on the same box) — consistent with the known host-round-trip cost per draft
round without on-device state save/restore. Not a bug report, just a data point:
the ggml-org#28118-class on-device checkpoint work matters more than kernels for MTP decode here.
Environment: Strix Halo (Ryzen AI MAX+ 395 / gfx1151, 128 GB, 96 GB carve), Windows 11,
TheRock ROCm 10.1.0 SDK, Clang 23, stock runtime (no retained-PM4), model
unsloth Qwen3.8-Flash-Next UD-IQ4_XS (not the PROJFIX quant),
-b/-ub 8192,--load-mode none(lazy-direct unavailable on Windows; PLE on the mmap fallback).1. Build fix needed (one line)
src/models/qwen4exp.cpp:1776callsple_reader->prefetch(...)unconditionally, but the_WIN32branch ofllama_lazy_reader(llama-lazy-reader.h) only stubsgather().Fix: add a no-op
void prefetch(const int32_t *, int64_t) const {}beside the stubbedgather (prefetch is a page-cache hint; no-op is behavior-correct on the mmap fallback).
2. Results with that fix (all correctness-gated: math probe + 24k needle, verbatim answers)
flood (
//////...), empty content. pp meanwhile 1080–1256 t/s (fast-but-wrong).flood (one ambiguous non-flood wrong-answer run, unreproduced).
LLAMA_MMB_HC16=0: fully clean — correct at 2k/8k/16k depth,needle exact at 24k — at pp 964–1045 t/s (262144 ctx), ~6x stock-mainline-class
prefill on this hardware and ~85-90% of the Linux reference numbers, with no custom
runtime and no lazy-direct.
Conclusion:
LLAMA_MMB_HC16(bf16 HC streams end-to-end) miscompiles or misbehaves incombination with the HC family on Windows/TheRock 10.1/Clang 23. Everything else ports.
Given its ablation worth is only 1.08x, defaulting it off on _WIN32 (or gating it on the
tested runtime) would make the branch Windows-viable as-is.
3. MTP on Windows (informational)
unsloth self-contained
mtp-...-Q8_0.ggufloads and drafts via--spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75: acceptance 77–79%,but net decode only ~24 t/s (vs ~34–38 on a branch carrying device-resident speculative
checkpoints on the same box) — consistent with the known host-round-trip cost per draft
round without on-device state save/restore. Not a bug report, just a data point:
the ggml-org#28118-class on-device checkpoint work matters more than kernels for MTP decode here.