No CI lane compiles a .hip: every src/vt/rocm change reaches main with zero build coverage, and #2957 landed that way

[localai-org-maint-bot]

Row: -

No lane in .github/workflows/ci.yml compiles a .hip file. The lanes are:

build-test-cpu            build-test-cpu-arm64      build-test-cpu-arm64-full
build-test-vulkan         cuda-arch-features        cuda-fat-build
windows-msvc-cpu          windows-msvc-vulkan       macos-metal-mlx
sanitize-cpu (address,undefined)                    sanitize-cpu (thread)
device-leakage            pr-size                   vulkan-spirv-freshness
drift-check               last-gated-commit         plan

So every change under src/vt/rocm/ — 30-odd translation units — reaches main with no build coverage of any kind. A green rollup on such a PR is not weak evidence; it is zero evidence, because nothing in it read the file.

This is not hypothetical. Both instances are from 2026-09-06.

Instance 1 — #3001 was held, and the manual gate was necessary

src/vt/rocm/rocm_moe_gate_up_swiglu.hip was written by an implementer whose host had no ROCm toolchain, so the TU had been compiled by nobody. Its CI rollup was green on the lanes that ran. I held the merge and queued a compile job on strix:gpu0 (4438b50e), which built it and ran the focused case:

NINJA rc=0
[557/585] Building HIP object .../rocm_moe_gate_up_swiglu.hip.o
SELECTED_CASES=1 ASSERTIONS=50

It happened to build. The point is that nothing in the project would have told us if it had not.

Instance 2 — #2957 landed unverified, and was compiled for the first time AFTERWARDS

ac77de7b4 merged the silu/gelu gate-dtype narrowing, touching src/vt/rocm/rocm_dense_basic.hip and src/vt/rocm/rocm_moe_router.hip. Its rollup carried one entry (plan).

A job queued for an unrelated attribution question (74a5a81c) built the merge commit on gfx1151 hours later and ran its focused case: 3 test cases, 7061/7061 assertions, 0 skipped, with a discrimination arm showing 6 bf16 := true CHECKs going red without the fix. Good result — and it arrived after the code was already on main. Had it not compiled, main would have been broken for every ROCm consumer with no gate anywhere to catch it.

This is #2950 with a specific, dated instance rather than a general worry.

What would close it, in increasing order of cost

1. A refusal, which needs no hardware. A checker that fails when a change touches src/vt/rocm/** and no ROCm build ran, pointing at the manual recipe. It converts silent absence into a visible PENDING, which is what AGENTS.md §Gates asks for when a gate cannot run. This alone would have flagged both instances above.
2. A compile-only lane, no GPU: install the ROCm toolchain in the runner and cmake -DVLLM_CPP_HIP=ON -DVLLM_CPP_HIP_ARCHITECTURES=gfx1151 + build. Catches every syntax and semantic error, which is what both instances were actually at risk of. It does not run a kernel and does not need a device.
3. A device lane on strix:gpu0, running the focused cross-device cases. Real coverage, but it needs a lease and is contended.

(2) is the one worth doing. The recipe is already proven — /mnt/nas_share/rc/rocmbuild-3001/job.sh configures and builds test_backend_cross_device for gfx1151 in minutes, and docs/bench-evidence/ carries several working ROCm build recipes.

One trap to carry into whatever is built

A doctest filter that matches no case exits 0 and prints Status: SUCCESS! over 0 test cases | 0 assertions. The first attempt at #3001's device gate did exactly that and would have merged it on a gate that measured nothing. Any lane added here must read both counts and treat a zero as VOID, and ideally derive the expected assertion count from the test source so a self-skip is distinguishable from a real run rather than merely non-zero.