matmul_bt_quant_grouped's 'the loader pre-filters, so reaching here is a bug' assertion is reachable: GLM-5.3-Flash hits it mid-forward on ROCm

[localai-org-maint-bot]

Row: -

src/vt/rocm/rocm_grouped_gemm.hip's matmul_bt_quant_grouped refuses an unsupported weight dtype with a message that asserts an invariant:

vt rocm: matmul_bt_quant_grouped: unsupported weight dtype
(ported: Q8_0/Q4_K/Q5_K/Q6_K; owed: Q4_0/Q2_K/Q3_K/IQ2_XXS/IQ3_XXS/IQ2_S/MXFP4
 -- the loader pre-filters, so reaching here is a bug)

We reached it. So the invariant does not hold.

What ran

rc job 0680eea4-195b-4df4-b4c1-19b7d04b3762 on strix:gpu0 (gfx1151, ROCm 7.2.4), 2026-09-06, on a tree carrying #3001 (93291d9cb). The real 103 GB unsloth/GLM-5.3-Flash-GGUF UD-Q2_K_XL, vllm-cli --model ... --device auto --prompt 'The capital of France is' --max-tokens 4 --temperature 0, with VT_GLM5_NEXT_DEVICE_EXPERTS=1 VT_CPU_MOE=1. Nothing reached the box by ssh.

engine: device placement INSTALLED: 45 layers run their routed experts on cpu, the rest on rocm
[glm5-next] the routed-expert keep-quant GEMM is running on DEVICE type 5 index 0;
  every other primitive of this model is on the host (spec section W9c-3a, O43, issue #2464).
[vt op-provider] op=113 device=5 selected=vt-native priority=0 registered=1 caps=0.0/unprobed
engine-fatal: EngineCore busy loop threw: vt rocm: matmul_bt_quant_grouped: unsupported weight dtype ...

rc=1, wall 2625 s, RAW BYTES CAPTURED: 2391. The control leg in the same job — same binary, same artifact, --device cpu — exited 0 and emitted Paris. Paris is, so the build and the artifact are sound and this refusal is a property of the ROCm path.

Why it is a bug and not just a missing encoding

Two separate things are true and only one of them is about encodings:

1. The ROCm grouped keep-quant GEMM supports four encodings and this artifact needs one it does not. That is the missing-arm half, and #2782 is the PR that adds the owed set.
2. The loader was supposed to prevent this call from ever being made, and it did not. src/vllm/model_executor/model_loader/gguf_keep_quant.cpp:136 returns true on kROCM for exactly Q8_0 || Q4_K || Q5_K || Q6_K, and the kernel's message says it relies on that pre-filter. Either the GLM-5.3-Flash routed-expert route does not consult that predicate at all, or it consults a different one, or it consults it against a different tensor than the one that reaches the GEMM.

Fixing (1) would hide (2) rather than fix it: once the owed encodings are ported, this particular artifact stops reaching the assertion, and the next model with an unported encoding walks into the same hole. The pre-filter is the thing that is supposed to turn "unsupported encoding" into a clean named refusal at load time instead of an engine-fatal mid-forward, and right now it does not do that on this path.

What would close it

Establish which of the three cases holds — not consulted, different predicate, or different tensor — and make the routed-expert device route consult DeviceKeepQuantSupported for the tensors it will actually hand the GEMM. The mutation that proves it: with the fix in place and an unported encoding, the run should refuse at load by name, and deleting the pre-filter call should put the engine-fatal back.

Do not resolve this by weakening the kernel's assertion. The message is correct about what it expects; the caller is what is wrong.

Provenance

Found while testing whether GLM-5.3-Flash reaches a ROCm queue after #3001 registered kMoeGateUpSwiGLUGrouped. It does — this is the next refusal in that chain, tracked for the model side under #2942.