hip: ROCm/HIP-only perf (TOP_K wave32 hybrid, k-quant VDR)

[raufaser]

ROCm/HIP-only performance work. Every hunk in this PR is #if defined(GGML_USE_HIP)- / RDNA-arch-gated, CPU-only, or test-only. The sgemm hunk is back to upstream after review, so CUDA/Vulkan/CPU behavior is byte-identical. Verified with a CUDA negative control: clean-v0.4.7 baseline vs baseline+this-PR -- test-backend-ops MUL_MAT and TOP_K all-pass in both states (identical outcomes: 1818/1818 each).

What's inside (all HIP/RDNA-gated)

1. TOP_K wave32-native + hybrid kernels (ggml-cuda/top-k.cu)
Idea from nasone32's RDNA3-7900xtx-opt ("Applied series"). Replaces the no-CUB HIP radix fallback with wave32-native tiled + hybrid kernels, keeping the CUB preference where CUB exists. Op-level: TOP_K ne=[200000,1,1,1] k=40 runs on GPU in ~86.7 us vs ~161.2 us on the CPU fallback path. Validation: test-backend-ops TOP_K 525/525 pass (ROCm0, gfx1100), same-seed decode byte-identical. Review fix: per-iteration seeding of the radix selection state, which stayed uninitialized (first pass) / stale (later passes) for all-+inf/NaN rows.

2. RDNA k-quant VDR decode boosts (ggml-cuda/mmvq.cu, ggml-cuda/vecdotq.cuh)
Idea from stew675's llama-cpp-rdna-boosts block 10 (RDNA-gated VDR bumps for Q4_K/Q5_K/Q6_K/Q8_0). Decode tg64 +4-6% on gfx1100. Validation: test-backend-ops MUL_MAT 1293/1293 pass (372 unsupported-type skips of 1665 selected cases), KLD identical vs the v0.4.7 base (see table), same-seed decode byte-identical. (The new *_vdr4/*_vdr2 entry points compile on all backends but are dead code outside the RDNA dispatch gate -- confirmed by the CUDA-build negative control.)

3. Small HIP-gated + neutral hunks

• ggml-cuda.cu: integrated=false device flag under GGML_USE_HIP (the CUDA #else arm already was false -- no behavior change). Review fix: the "at no cost" comment now records the placement trade-off, with a matching note in docs/beellama-features.md (APU tensors move off host-mapped memory, VRAM headroom changes, unmeasured -- no APU hardware here).
• ggml-cpu/llamafile/sgemm.cpp: REVERTED to upstream if (n < 2) per review -- the CPU neutrality claim above is exact again.
• tests/test-backend-ops.cpp: extra Q6_K/Q4_K/Q5_K + FA-head tuning shapes (test-only); removed one duplicated dead Q8_0 line.

Validation (gfx1100 RX 7900 XTX + RTX 4090 CUDA gate)

• HIP test-backend-ops: TOP_K 525/525, MUL_MAT 1293/1293 (ROCm0).
• CUDA negative control (RTX 4090, sm_89): baseline vs +PR builds produce identical test-backend-ops results (1818/1818 in both, byte-identical outcomes) plus ctest -R "kvarn|fattn-route" 15/15 -- this PR provably does not change CUDA behavior.
• KLD trio (kvarn6, -c 512 -ub 512, ngl99), stock v0.4.7 / this PR / PR hip: shared perf (MMQ MoE tiles + mmvdq, prefill plumbing) #154 -- bit-identical across all three trees:

model	KLD	same-top
4B Q4_K_M	2.197615	61.961%
27B Q5_K_S	0.004916	95.490%
35B-A3B	0.004116	97.843%

• The 4B leg matches stock v0.4.7 bit-for-bit (pre-existing base behavior -- v0.4.7 lacks PR Make KVarN usable on ROCm RDNA3 (toward #122: D256 race, portable prefill, f32 WMMA prototype) #149's KVarN fixes; Make KVarN usable on ROCm RDNA3 (toward #122: D256 race, portable prefill, f32 WMMA prototype) #149-final measures 0.006239/97.84%). 4B with plain f16 cache is clean on this tree (0.006621/96.275%), exonerating the VDR matmul path. 27B/35B sit next to the Make KVarN usable on ROCm RDNA3 (toward #122: D256 race, portable prefill, f32 WMMA prototype) #149-final figures (0.005708/94.51%, 0.003861/97.45%).
• Bench 27B Q5_K_S kvarn6: pp512 ~466, pp4096 ~173, tg ~34-36 -- neutral vs stock v0.4.7 (same numbers). Make KVarN usable on ROCm RDNA3 (toward #122: D256 race, portable prefill, f32 WMMA prototype) #149-final differs (pp4096 ~650) through its KVarN prefill fast path. Full performance needs PR Make KVarN usable on ROCm RDNA3 (toward #122: D256 race, portable prefill, f32 WMMA prototype) #149 + hip: ROCm/HIP-only perf (TOP_K wave32 hybrid, k-quant VDR) #153 + hip: shared perf (MMQ MoE tiles + mmvdq, prefill plumbing) #154 together; each PR is neutral-or-better vs its own base on its own.
• test-kvarn ladder aborts at the head-wide store-route require -- pre-existing on the v0.4.7 base (fixed in Make KVarN usable on ROCm RDNA3 (toward #122: D256 race, portable prefill, f32 WMMA prototype) #149 round 2; Make KVarN usable on ROCm RDNA3 (toward #122: D256 race, portable prefill, f32 WMMA prototype) #149-final passes).

Kudos

• nasone32 (llama.cpp-RDNA3-7900xtx-opt) for the wave32 TOP_K hybrid design this ports.
• stew675 (llama-cpp-rdna-boosts, block 10) for the RDNA k-quant VDR tuning this ports.
• Porting, validation, and CUDA negative control by raufaser; numbers measured on RX 7900 XTX (gfx1100) unless noted.

Base: v0.4.7. Sibling PR (shared-code perf) follows separately as PR B.

[Anbeeld]

Requesting changes on the CPU dispatch hunk; the rest are non-blocking.

Blocking: ggml/src/ggml-cpu/llamafile/sgemm.cpp:3826 (was if (n < 2) at 3819). Changing the guard to n <= 8 is not HIP-gated. It runs at the top of llamafile_sgemm before the type switch, GGML_LLAMAFILE is on by default, and __MMA__ is never defined on x86-64 or ARM, so every CPU build now sends n = 2..8 to the generic vec_dot path instead of tinyBLAS (ggml-cpu.c:1342,1417). That range is what a speculative verify batch uses (n_draft+1), and it is also what -b/-ub in 2..8 selects for prompt processing. The PR body says CPU behavior is byte-identical, which cannot hold for this hunk. Please either point at the fork requirement or test that needs batch-versus-sequential bit-identity (I could not find one; tools/server/README.md:530 documents that logits are not bit-identical across batch sizes), scope the change to that validation path, or move it to the shared PR with CPU measurements. If it stays, drop the neutrality claim and record the n = 2..8 CPU effect.

Non-blocking:

• ggml/src/ggml-cuda/top-k.cu:245-256: selected_bucket and selected_total are written only when __ffsll(...) - 1 matches a lane. If both ballots are zero, which happens when the current range holds fewer than limit keys (an all-+inf/NaN logits row, since ordered keys at or above 0xff800000 are excluded), then bucket = selected_bucket reads uninitialized shared memory on the first pass or stale values on later passes. The outcome is wrong indices, not memory unsafety. Seeding the state per iteration would close it.
• ggml/src/ggml-cuda/ggml-cuda.cu:315: integrated = false on HIP also changes placement, because ggml_backend_cuda_device_supports_buft (ggml-cuda.cu:5827) gates the CUDA-host buffer type on that field. On APUs this can move tensors off host-mapped memory and change VRAM headroom, so "at no decode/prefill cost" understates it. Worth recording in docs/beellama-features.md next to the measured PPL 5.9243 to 8.51+ corruption.
• tests/test-backend-ops.cpp:10086-10087: the Q8_0 17408x512 line is added twice, inside the #if 0 region at 10075-10086, so it is dead code.

What I checked: the new Q4_K/Q5_K VDR=4 and Q6_K VDR=2 entry points match the canonical VDR=2/VDR=1 element partition, the q8_1 block pairing and scale/d8 indices, and the kqs = vdr * (tid % (qi/vdr)) stride. No builds or tests were run, so the perf and byte-identity numbers in the description are unverified here.

[raufaser]

All items addressed in 5344385 (on top of abe3026, no force-push). Builds + validation below are new runs on the final tree: HIP = RX 7900 XTX gfx1100 (TheRock, clean GPU_TARGETS=gfx1100 configure), CUDA = RTX 4090 sm_89.

Blocking (sgemm): reverted to upstream if (n < 2) -- the batch-vs-seq comment block is gone, so the neutrality claim in the body is exact again. No CPU measurement needed (upstream path restored); the CUDA negative control below proves no impact anywhere.

• top-k.cu: per-iteration seeding of selected_bucket/selected_total before the ballot section. Safe for normal rows (any nonzero ballot overwrites before the broadcast read). test-backend-ops -o TOP_K: 525/525 pass on ROCm0.
• ggml-cuda.cu: the "at no cost" claim is struck -- the comment now records that supports_buft gates the CUDA-host buffer type on integrated, so APU tensors move off host-mapped memory and VRAM headroom changes (unmeasured, no APU hardware here). Matching note added to docs/beellama-features.md next to the HIP APU paragraph.
• test-backend-ops.cpp: deleted one copy of the duplicated dead Q8_0 line.

Validation table:

• HIP backend-ops: TOP_K 525/525, MUL_MAT 1293/1293 (372 unsupported-type skips of 1665 selected).
• CUDA negative control: clean-v0.4.7 baseline vs +PR builds give byte-identical outcomes -- MUL_MAT+TOP_K 1818/1818 in both -- plus ctest -R "kvarn|fattn-route" 15/15 on the +PR tree.
• KLD trio (kvarn6, -c 512 -ub 512, ngl99) is bit-identical across stock v0.4.7, this PR, and PR hip: shared perf (MMQ MoE tiles + mmvdq, prefill plumbing) #154: 4B 2.197615/61.961%, 27B 0.004916/95.490%, 35B 0.004116/97.843%. The 4B leg reproduces bit-for-bit on the base commit without my changes (separate clean-tree control build), so it predates this PR: v0.4.7 lacks PR Make KVarN usable on ROCm RDNA3 (toward #122: D256 race, portable prefill, f32 WMMA prototype) #149's KVarN fixes. 4B with plain f16 cache is clean on this tree (0.006621/96.275%), exonerating the VDR matmul path.
• Bench 27B kvarn6: pp512 ~466, pp4096 ~173, tg ~34-36 -- neutral vs stock v0.4.7. Make KVarN usable on ROCm RDNA3 (toward #122: D256 race, portable prefill, f32 WMMA prototype) #149-final differs (pp4096 ~650) through its KVarN prefill fast path. Full performance needs Make KVarN usable on ROCm RDNA3 (toward #122: D256 race, portable prefill, f32 WMMA prototype) #149 + hip: ROCm/HIP-only perf (TOP_K wave32 hybrid, k-quant VDR) #153 + hip: shared perf (MMQ MoE tiles + mmvdq, prefill plumbing) #154 together (noted in the body); each PR is neutral-or-better vs its own base standalone.
• test-kvarn ladder aborts at the head-wide store-route require -- pre-existing on the v0.4.7 base (fixed in Make KVarN usable on ROCm RDNA3 (toward #122: D256 race, portable prefill, f32 WMMA prototype) #149 round 2; Make KVarN usable on ROCm RDNA3 (toward #122: D256 race, portable prefill, f32 WMMA prototype) #149-final passes "nkv ladder OK").

The only build failure is the known pre-existing test-kv-history.exe link error on Windows (documented v0.4.7 quirk, also fails on the reference build); everything else links, including all validation binaries.