fix(mq4v2): one MQ-V2 prefill admit rule for llama and qwen35; discriminating GEMM parity; spec §9 - #690
fix(mq4v2): one MQ-V2 prefill admit rule for llama and qwen35; discriminating GEMM parity; spec §9#690Kaden-Schutt wants to merge 5 commits into
Conversation
llama::is_batchable_la admitted MQ4G256V2/MQ6/5/3/2G256V2 for WMMA
prefill only on gfx1200/gfx1201 while qwen35::is_batchable_la admitted
them on gfx11+gfx12 behind HIPFIRE_MQV2_GFX11_WMMA, so plain Llama/Qwen3
dense qt=44 models prefetched per-token on gfx1100/1151 while Qwen3.5/3.8
took WMMA — despite both doc-comments claiming an exact match (audit
2026-09-02 Broken 1).
Move the dtype set + arch set + kill-switch helper into
llama::mqv2_wmma_batchable / llama::mqv2_gfx11_wmma_enabled_from_env in
hipfire-runtime and delegate from both callers, so the lockstep is
structural. MQ4CG256 (qt=45) stays gfx12-only in both by intent.
Tests: rename the two gfx12-only llama admit tests to gfx11+gfx12
expectations, repoint qwen35 env-escape test at the shared helper, and
add mqv2_admit_llama_qwen35_lockstep asserting both gates agree over
the MQ-V2 dtypes x {gfx1100, gfx1151, gfx1201, gfx1030, gfx1010}.
…ections mq4v2_gemm_parity's Gaussian weights give both halves near-identical headers, so a wrong half-select hides in quantization noise despite the header comment promising a systematic blow-up (audit 2026-09-02 Missing 1). Add arm 2 using the disjoint-halves construction from mq4v2_residual_parity (half0 [-1,1], half1 [96,160]) over the same batch-size sweep: v2 output must match the f32 reference within 5% rel-RMS, and the swapped-headers negative control (as in mq4v2_moe_parity) must DISAGREE. Keep the Gaussian v1-vs-v2 arm and fix the header comment to state what each arm can and cannot detect. Add a host-side test proving the fixture discriminates with no GPU. Docs (audit Would-change 1-2): spec section 9 now records MoE as production-wired for qt=44, the XBATCH single-row path as ported, and the gfx11 kt+=2 / residual kt++ stepping; residual_mmq.hip loses its stale Experimental tag; the gfx12 QKV kernel loses its HYPOTHESIS/scaffold wording for the validated C-map statement.
9b456ea to
8d4450b
Compare
hw-gate sol prelimsummary: Centralizes the MQ-V2 dtype/architecture/kill-switch predicate used by Qwen3.5-family batched prefill, while deliberately changing plain Llama/Qwen3 MQ-V2 on gfx12 from batched prefill to per-token fallback because its chunk path lacks V2 dispatch arms. It also adds a disjoint-half MQ4V2 GEMM parity fixture with a swapped-header negative control and updates kernel/spec comments; the changed HIP files contain comments only. run_hardware: true routes:
unavailable_routes: (none) claim_assessment: The author claims Qwen3.5-family MQ-V2 admission is correct on gfx11/gfx12, plain-Llama V2 must fall back everywhere, and the GEMM fixture now detects half-header selection errors. Coherent qwen3.8 MQ4-XT battery/chain output can evidence the available real-artifact load and gfx12 Qwen3.5 path. The plain-Llama fallback claim is not fully provable without a plain-Llama qt44/47-50 fixture, and the discriminating kernel claim requires the actual mq4v2 GEMM parity output rather than unit-test counts or PR prose. questions_for_author:
|
hw-gate evidence — 2 lane(s) — verdict passlane hiptrx (gfx1201)hw-gate evidence
fixturesqwen3.6:27bsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 33.4 status pass
qwen3.6:27b battery turn 0qwen3.6:27b battery turn 1qwen3.6:27b battery turn 2qwen3.6:27b battery turn 3qwen3.6:27b battery turn 4ornith-1.5:35b-a3b-mq4rsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 32.7 status pass
ornith-1.5:35b-a3b-mq4r battery turn 0ornith-1.5:35b-a3b-mq4r battery turn 1ornith-1.5:35b-a3b-mq4r battery turn 2ornith-1.5:35b-a3b-mq4r battery turn 3ornith-1.5:35b-a3b-mq4r battery turn 4lfm2.5:1.2bsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 8.1 status pass
lfm2.5:1.2b battery turn 0lfm2.5:1.2b battery turn 1lfm2.5:1.2b battery turn 2lfm2.5:1.2b battery turn 3lfm2.5:1.2b battery turn 4qwen3.8:27b-mq4-xtsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 11.7 status pass
qwen3.8:27b-mq4-xt battery turn 0qwen3.8:27b-mq4-xt battery turn 1qwen3.8:27b-mq4-xt battery turn 2qwen3.8:27b-mq4-xt battery turn 3qwen3.8:27b-mq4-xt battery turn 4chain — exit 0 seconds 21.1 status pass
qwen3.8:27b-mq4-xt chain turn 0qwen3.8:27b-mq4-xt chain turn 1qwen3.8:27b-mq4-xt chain turn 2qwen3.8:27b-mq4-xt chain turn 3qwen3.8:27b-mq4-xt chain turn 4kernelstatus: pass report pass: True lane hipx (gfx1100)hw-gate evidence
fixturesqwen3.6:27bsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 32.7 status pass
qwen3.6:27b battery turn 0qwen3.6:27b battery turn 1qwen3.6:27b battery turn 2qwen3.6:27b battery turn 3qwen3.6:27b battery turn 4ornith-1.5:35b-a3b-mq4rsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 33.7 status pass
ornith-1.5:35b-a3b-mq4r battery turn 0ornith-1.5:35b-a3b-mq4r battery turn 1ornith-1.5:35b-a3b-mq4r battery turn 2ornith-1.5:35b-a3b-mq4r battery turn 3ornith-1.5:35b-a3b-mq4r battery turn 4lfm2.5:1.2bsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 8.1 status pass
lfm2.5:1.2b battery turn 0lfm2.5:1.2b battery turn 1lfm2.5:1.2b battery turn 2lfm2.5:1.2b battery turn 3lfm2.5:1.2b battery turn 4qwen3.8:27b-mq4-xtsource: bucket sha256_ok: ✅ size_ok: ✅ status: pass reason: battery — exit 0 seconds 22.1 status pass
qwen3.8:27b-mq4-xt battery turn 0qwen3.8:27b-mq4-xt battery turn 1qwen3.8:27b-mq4-xt battery turn 2qwen3.8:27b-mq4-xt battery turn 3qwen3.8:27b-mq4-xt battery turn 4chain — exit 0 seconds 24.4 status pass
qwen3.8:27b-mq4-xt chain turn 0qwen3.8:27b-mq4-xt chain turn 1qwen3.8:27b-mq4-xt chain turn 2qwen3.8:27b-mq4-xt chain turn 3qwen3.8:27b-mq4-xt chain turn 4kernelstatus: pass report pass: True |
hw-gate sol verdict{
"claim_verdict": "not-exercised",
"confidence": 0.93,
"coverage": {
"gaps": [
"No plain-Llama/Qwen3 qt44/47-50 artifact exercised the new all-architecture fallback in crates/hipfire-runtime/src/llama.rs.",
"The added mq4v2_gemm_parity disjoint-halves arm was not run in hw-gate.json; the Redline kernel report used qwen3.6-27b.mq4 and does not provide the required direct MQ4V2 half-header parity output.",
"No base-versus-head measurement quantifies the expected gfx12 plain-Llama prefill performance regression caused by replacing batched prefill with per-token fallback."
],
"surfaces_evidenced": [
"load",
"qwen35 MQ-V2 prefill on gfx1100 and gfx1201",
"decoded generation coherence"
],
"surfaces_touched": [
"kernel",
"load",
"prefill",
"docs"
]
},
"decision": "needs-human",
"eyeball": [
"qwen3.8:27b-mq4-xt battery outputs on gfx1100 and gfx1201 are coherent across code, science, factual recall, JSON, and arithmetic prompts; all expected substrings are present and no attractors, empty responses, special-token leakage, or runaways appear.",
"qwen3.8:27b-mq4-xt chain outputs on both architectures remain coherent through increasing context and related turns, including valid merge code and the correct 210-mile calculation.",
"The unusually slow first battery prefill on several fixtures is consistent with cold initialization; subsequent turns recover and decoded text remains coherent."
],
"phase": "verdict",
"rationale": "The available MQ4-XT artifact validates the Qwen3.5-family admission path on both gfx1100 and gfx1201: battery and chain pass with coherent decoded text. Load behavior is also exercised. However, the behavioral change at crates/hipfire-runtime/src/llama.rs:1917 is untested because no plain-Llama/Qwen3 V2 fixture ran, and the central discriminating-kernel claim is unsupported because hw-gate.json contains no mq4v2_gemm_parity output or swapped-header negative-control results. The generic Redline pass uses qwen3.6-27b.mq4 and cannot substitute for direct MQ4V2 parity. Human review is therefore required before accepting the uncovered fallback and parity claims.",
"regressions": []
}Floor: hard=[] soft=["coverage_gaps: ['No plain-Llama/Qwen3 qt44/47-50 artifact exercised the new all-architecture fallback in crates/hipfire-runtime/src/llama.rs.', 'The added mq4v2_gemm_parity disjoint-halves arm was not run in hw-gate.json; the Redline kernel report used qwen3.6-27b.mq4 and does not provide the required direct MQ4V2 half-header parity output.', 'No base-versus-head measurement quantifies the expected gfx12 plain-Llama prefill performance regression caused by replacing batched prefill with per-token fallback.']", 'model needs-human'] model_decision=needs-human final=needs-human |
|
announcement: Both lanes pass and I closed the three gaps Sol flagged. A base-vs-PR Redline A/B on qwen3.8:27b-mq4-xt (gfx1201) is bit-identical — same prefill/decode launch sequences and the same logits/KV/recurrent hashes — so the qwen35 delegation is a true no-op, and HIPFIRE_MQV2_GFX11_WMMA=0 correctly leaves gfx12 alone. I ran the PR's mq4v2_gemm_parity on gfx1201: the disjoint-halves arm passes (v2 rel-rms ~2-3e-4) with the swapped-header control disagreeing by 0.5-1.6, so the negative control is doing real work. The plain-Llama refusal can't be exercised on hardware because no arch 0/1 MQ-V2 artifact exists in the registry, but I traced forward_prefill_chunk myself (llama.rs:2577-2605, :2731-2745) and agree master's gfx12 admission skipped the rotate and ran V1 launchers on V2 blobs — the refusal fixes a silent-corruption hole rather than removing working behavior. One small follow-up when convenient: the new comment in gemm_mq4g256v2_residual_mmq.hip cites gemm.rs:17860-17909 for the batch>=128 gate, but that is the launcher helper; the actual gate is at the WMMA wrapper call sites (e.g. gemm.rs:28029-28036) and also excludes replay-recording and graph-capture. Merging to staging. override: Override Sol needs-human: I closed Sol's three gaps on this host. (1) The added mq4v2_gemm_parity disjoint-halves arm was run on gfx1201 from the PR build and PASSES with a load-bearing negative control (v2 rel-rms 1.8e-4..3.1e-4 vs swapped-header reference 0.53..1.62 across batch 1/8/12/16/32). (2) The qwen35 admission change is proven a no-op by a base-vs-PR Redline A/B on the real MQ4G256V2 artifact qwen3.8:27b-mq4-xt on gfx1201: prefill-128/512 and decode launch sequences, launch counts, and logits/KV/recurrent output hashes are bit-identical, and HIPFIRE_MQV2_GFX11_WMMA=0 leaves gfx12 untouched as specified. (3) The plain-Llama V2 refusal cannot be exercised on any host because no arch 0/1 MQ-V2 artifact exists in the registry (the only arch-1 entries are qwen3:0.6b/8b .hf4); but the source trace is unambiguous that master's gfx12 admission ran the V1 hfq4g256 launchers on V2 blobs with the FWHT rotate skipped (llama.rs:2577-2605, :2731-2745), so the PR converts a silently-incorrect path into per-token decode and no working behavior is lost. A 'perf regression' measurement against a path that produced wrong activations is not meaningful. investigation: | question | route | result | evidence | |---|---|---|---| | Does the qwen35 delegation to llama::mqv2_wmma_batchable change MQ-V2 prefill dispatch or output on gfx12? (A/B master vs PR on the real qt=44 fixture) | HIP_VISIBLE_DEVICES=1 python3 scripts/redline_daemon_harness.py --model $HIPFIRE_MODELS_DIR/qwen3.8-27b.mq4-xt --daemon $HW_GATE_BASE_BIN/daemon --out $HW_GATE_EVIDENCE/redline-qwen38-27b-mq4xt-gfx1201-BASE.json --capture-repeats 2 --measure-repeats 2 --decode-iterations 16 --timeout 300 ; then identical with --daemon $HW_GATE_BIN/daemon → ...-PR.json | Bit-identical across base and PR: prefill128 hash 2446cb9fe53c0779 (1140 launches), prefill512 hash 04eea88a8cd26904 (2280 launches = 2 chunks of 256, i.e. batched WMMA prefill, not per-token), decode hash 9afd3ae37c853481 (899 launches), shadow bit_exact=true with identical logits e3db4bf8eea6d23a / kv 81cdc593a6c30367 / recurrent c45b226d89a1dcf5. Decode capture names fused_qkv_mq4g256v2, fused_qkvza_mq4g256v2, gemv_mq4g256v2_residual — the fixture is genuinely MQ4G256V2. Only kernarg pointer values differ (different processes). prefill512 median 862.0 vs 857.4 tok/s, decode 36.1 vs 36.2. | .stderr.log) | | Does the shared rule's kill-switch plumbing (HIPFIRE_MQV2_GFX11_WMMA=0) leave gfx12 admission untouched end-to-end through hipfire_config::developer_var? | HIP_VISIBLE_DEVICES=0 HIPFIRE_MQV2_GFX11_WMMA=0 python3 scripts/redline_daemon_harness.py --model $HIPFIRE_MODELS_DIR/qwen3.8-27b.mq4-xt --daemon $HW_GATE_BIN/daemon --out $HW_GATE_EVIDENCE/redline-qwen38-27b-mq4xt-gfx1201-PR-killswitch0.json --capture-repeats 2 --measure-repeats 2 --decode-iterations 16 --timeout 300 | Identical to the PR default arm: same prefill/decode sequence hashes, same launch counts, shadow bit_exact=true, same output hashes. gfx12 is unaffected by the gfx11 kill-switch, matching llama.rs:1849-1861. | redline-qwen38-27b-mq4xt-gfx1201-PR-killswitch0.json | | Does the new disjoint-halves GEMM parity arm actually discriminate a half-select bug, and does the live v2 residual WMMA kernel pass it? (Sol gap 2) | HIP_VISIBLE_DEVICES=2 $HW_GATE_BIN/examples/mq4v2_gemm_parity (PR build, md5 e3b815787fd53ef18a579200f7ca1c6c, contains the arm-2 strings; head 4e360fb) | PASS, exit 0. Disjoint arm: batch 1/8/12/16/32 v2 rel-rms 1.85e-4 / 3.07e-4 / 3.03e-4 / 2.70e-4 / 2.50e-4 against the exact-dequant reference, while the swapped-header negative control disagrees by 5.35e-1 / 1.62 / 1.32 / 1.22 / 1.26 — three to four orders of magnitude apart, so the control is load-bearing. Gaussian arm unchanged (v2 2.7e-4 vs v1 8-9.6e-4); gate_up v2 2.62e-4; qkvza v2 2.62e-4 on all four outputs. | mq4v2_gemm_parity-gfx1201-PR.log | | Is the author's justification for refusing MQ-V2 in llama::is_batchable_la true — does llama.rs::forward_prefill_chunk really lack V2 arms so that master's gfx12 admission ran V1 launchers on V2 blobs? And who is actually affected? | Source trace of PR head and unproven:
rationale: The PR has exactly one behavior change and one refactor. The refactor — qwen35::is_batchable_la (prefill.rs:1543-1560) delegating to llama::mqv2_wmma_batchable (llama.rs:1871-1880) — is proven a no-op on gfx1201 by a base-vs-PR Redline A/B on the real MQ4G256V2 fixture qwen3.8:27b-mq4-xt (sha256 9f91556f…, size 14980361216): prefill-128/512 sequence hashes, launch counts (1140/2280, i.e. two 256-token chunks of batched WMMA prefill), decode sequence hash, and logits/KV/recurrent output hashes are identical between $HW_GATE_BASE_BIN/daemon and $HW_GATE_BIN/daemon ($HW_GATE_EVIDENCE/ab-summary-qwen38-mq4xt-gfx1201.json), and HIPFIRE_MQV2_GFX11_WMMA=0 leaves gfx12 unchanged. The mandatory lanes add coherent battery+chain output for the same fixture on gfx1100 and gfx1201. The behavior change — llama::is_batchable_la (llama.rs:1917-1947) refusing MQ4G256V2/MQ4CG256/MQ6-2G256V2 on every arch instead of admitting them on gfx12 — only affects arch_id 0/1 (carriers.rs:733-737), for which the registry carries no MQ-V2 artifact (qwen3:0.6b/8b are .hf4). I verified the author's justification directly: forward_prefill_chunk's matchers (llama.rs:2577-2580, :3032, :3124, :3255) list only V1 dtypes, so an admitted V2 layer skips the FWHT rotate and lands on gemm_qkv_hfq4g256 / gemm_hfq4g256_residual / gemm_gate_up_hfq4g256 — master's gfx12 path for that class was silently wrong, and the PR moves it to per-token decode. That is a fail-closed rule that refuses no real artifact and replaces incorrect behavior, so it is not a regression under the master-is-the-oracle standard. The new mq4v2_gemm_parity disjoint-halves arm was run from the PR build on gfx1201 and passes with a negative control that separates correct and swapped-header decodes by three to four orders of magnitude ($HW_GATE_EVIDENCE/mq4v2_gemm_parity-gfx1201-PR.log). The two .hip changes are comment-only and the daemon is untouched. One doc nit for the author: gemm_mq4g256v2_residual_mmq.hip:12-14 cites gemm.rs:17860-17909 for the batch>=128 && batch%128==0 selection, but that range is the launcher helper; the gate lives at the WMMA wrapper call sites (e.g. gemm.rs:28029-28036) and also requires not-recording and not-capturing. To reverse this decision a maintainer would need either an arch 0/1 MQ-V2 artifact that decodes coherently through master's batched path on gfx12 (the source says it cannot), or a gfx11 A/B showing the qwen35 sequence hash changed. merged: beta d14b3a0 soft floor: ["coverage_gaps: ['No plain-Llama/Qwen3 qt44/47-50 artifact exercised the new all-architecture fallback in crates/hipfire-runtime/src/llama.rs.', 'The added mq4v2_gemm_parity disjoint-halves arm was not run in hw-gate.json; the Redline kernel report used qwen3.6-27b.mq4 and does not provide the required direct MQ4V2 half-header parity output.', 'No base-versus-head measurement quantifies the expected gfx12 plain-Llama prefill performance regression caused by replacing batched prefill with per-token fallback.']", 'model needs-human'] |
There was a problem hiding this comment.
hw-gate sol verdict needs-human: The available MQ-V2-adjacent fixture qwen3.8:27b-mq4-xt loaded and generated coherent battery and chain output on both gfx1100 and gfx1201, and the Redline harness passed stable capture plus bit-exact HIP/AQL parity. However, the principal behavior change at crates/hipfire-runtime/src/llama.rs:1916 newly admits plain Llama/Qwen3 MQ-V2 WMMA prefill on gfx11, and no artifact exercising that architecture path was run. The dedicated disjoint-halves GPU parity example was also absent from the gate evidence. These are material coverage gaps for a kernel-admission change, requiring human disposition rather than a greenlight.
There was a problem hiding this comment.
Two of the three gaps Sol flagged closed clean on gfx1201: your parity example passes here with the disjoint arm discriminating exactly as you reported (2.5e-4 vs 1.263 at batch 32), and a base-vs-PR redline A/B on qwen3.8-27b.mq4-xt is bit-identical down to the logits hash, with the prefill capture naming gemm_qkv/qkvza/gate_up/residual_mq4g256v2_wmma_gfx12 as the executed path. The blocker is the lockstep change itself: llama.rs forward_prefill_chunk has no MQ-V2 arms (matchers at 2570-2573 / 3025-3028 / 3117-3120 / 3248-3251 list MQ4G256|MQ6G256|MQ3G256|MFP4G32 only), so a plain Llama/Qwen3-dense qt44/47-50 model admitted on gfx11 now skips the FWHT rotate and lands on gemm_qkv_hfq4g256 / gemm_hfq4g256_residual / gemm_gate_up_hfq4g256 — the V1-on-V2 mis-decode your own comment at llama.rs:1501-1507 describes — where master ran it per-token and correct. Either give the llama chunk path real V2 arms and show a --format mq4 qwen3-8b on gfx1100 through serve_harness + redline, or keep the shared rule but have llama exclude MQ-V2 until then (that also fixes master's existing gfx12 admission) and make the lockstep test assert the intended asymmetry. The parity arm and doc corrections are good and should come back with the fix; evidence is in fable-evidence/, see README.md and STATIC-llama-batched-prefill-no-mqv2-arms.md. Hard floor: clear. Mandatory routes: all 4 fixtures x 2 lanes pass with coherent decoded turns, and the hw-gate redline route passed on qwen3.6-27b.mq4 (a V1 fixture, so it did not touch the dtype this PR is about). What I added: (a) redline on the actual MQ-V2 fixture qwen3.8-27b.mq4-xt on gfx1201, PR vs master — sequence hashes and logits/kv/recurrent hashes bit-identical (redline-mq4v2xt-gfx1201-AB-summary.txt), so on gfx12 this PR changes nothing for the qwen35 path, as the diff predicts; (b) the prefill kernel names for that fixture, which are the gemm__mq4g256v2_wmma_gfx12 family (Sol gap 3 closed); (c) the PR's mq4v2_gemm_parity example on gfx1201, PASS with the disjoint arm at 1.8e-4..3.1e-4 vs 0.53..1.62 swapped (Sol gap 2 closed). The block is item 1 of the PR body: 'plain Llama/Qwen3 dense models on MQ-V2 now take WMMA prefill on gfx11 (they were per-token)'. That admission leads into crates/hipfire-runtime/src/llama.rs forward_prefill_chunk, which has no MQ-V2 branch in any of its four GEMM chains (matchers 2570-2573, 3025-3028, 3117-3120, 3248-3251; fallthroughs 2725, 3105, 3236, 3335 to hfq4g256 V1 launchers; no rotate at 2590). llama.rs:1501-1507 spells out the result of routing a V2 blob to a V1 launcher. The lockstep premise — same admit rule for llama and qwen35 — is only sound if both batched bodies dispatch the same dtype set; qwen35/prefill.rs does (hence the clean gfx1100/gfx1201 lanes on the arch-5 fixture), llama.rs does not. The audit's 'Broken 1' was two crates disagreeing; the fix chosen makes them agree on the side that is wrong for llama. Correct options: (1) add MQ-V2 arms to forward_prefill_chunk (rotate + gemm_qkv_mqg256v2_wmma / gemm_gate_up_mqg256v2_wmma / gemm_mqg256v2_residual_wmma via the same rdna-compute entry points qwen35 uses, per the scout table) and evidence it on gfx1100 with a --format mq4 qwen3-8b through serve_harness battery + redline; or (2) keep mqv2_wmma_batchable shared but have llama::is_batchable_la exclude MQ-V2 on every arch until (1) lands — which also closes master's pre-existing gfx12 hole — and rewrite mqv2_admit_llama_qwen35_lockstep to assert the intended asymmetry instead of equality. Items 2 and 3 of the PR (parity arm, doc/comment corrections) are verified and fine to carry into the resubmission. Registry impact today is nil (no arch-0/1 MQ-V2 artifact), which is why nothing in the gate could see this; that is an argument for why it slipped, not for merging it. Override Sol needs-human: Sol's three coverage gaps split two ways once I ran them. Gaps 2 and 3 closed clean on this host: the PR's parity example passes on gfx1201 with the disjoint-halves arm discriminating (v2 rel-rms 1.8e-4..3.1e-4 vs swapped-header 0.53..1.62), and a base-vs-PR redline A/B on qwen3.8-27b.mq4-xt is bit-identical (same prefill/decode sequence hashes, same logits/kv/recurrent hashes) with the prefill capture naming gemm_qkv/qkvza/gate_up/residual_mq4g256v2_wmma_gfx12 as the executed path. Gap 1 — the PR's headline behaviour change — is not a coverage gap; it is a defect. llama.rs's batched prefill body (forward_prefill_chunk) has no MQ-V2 arms at all: the per-layer matchers at llama.rs:2570-2573/3025-3028/3117-3120/3248-3251 list MQ4G256|MQ6G256|MQ3G256|MFP4G32 only, so a *G256V2 layer gets rmsnorm_batched (no FWHT) and the else arms gemm_qkv_hfq4g256 (2725), gemm_hfq4g256_residual (3105, 3335), gemm_gate_up_hfq4g256 (3236) — V1 launchers on V2 blobs, the failure llama.rs:1501-1507 itself describes as 'bit_casts an fp16 pair to f32 and decodes every weight to ~1e-14 — no error, full speed, pure noise'. On master, arch-0/1 MQ-V2 on gfx11 is not eligible (llama.rs:2181-2208) and takes the per-token path whose GEMV arms (1339-1349, 1418-1428) decode V2 correctly. The PR turns correct-but-slow into silently wrong for that class. More hardware evidence cannot fix this; the code must change, so block rather than needs-human.
…efill_chunk has no V2 arms hw-gate Fable seat on #690 (run 33895641944), source trace verified: the shared MQ-V2 admit rule made llama::is_batchable_la admit plain Llama/Qwen3-dense qt44/47-50 artifacts to WMMA prefill on gfx11 and gfx12, but llama.rs::forward_prefill_chunk's per-layer matchers (qkv_is_mq ~:2570, wo_is_mq ~:3025, ffn_is_mq ~:3117, w_down_is_mq ~:3248) list only MQ4G256|MQ6G256|MQ3G256|MFP4G32 — an admitted V2 model skips the FWHT rotate and runs the V1 hfq4g256 launchers on V2 blobs: silently incoherent prefill. master's pre-existing mq4_v2_gfx12 arm had the same hole on gfx12; no gfx12 Llama-V2 artifact has ever tripped it. llama::is_batchable_la now refuses every *G256V2 dtype and MQ4CG256 on every arch, with the reason at the site. qwen35::is_batchable_la keeps the shared mqv2_wmma_batchable rule (its chunk path has the V2 arms; gfx11 kill-switch intact). llama_spec::batched_verify_eligible routes all seven weights through is_batchable_la, so it is covered without an edit. Lockstep test now asserts the true contract: agreement on every non-V2 dtype across 5 arches; for V2, qwen35 admits on gfx11/gfx12 and llama refuses everywhere. Spec §9 row and crate maps corrected. hipfire-runtime is_batchable_la: 7 passed; qwen35 is_batchable + lockstep: 9 passed.
|
Fable's finding on run 33895641944 is fixed in |
There was a problem hiding this comment.
hw-gate sol verdict needs-human: The available MQ4-XT artifact validates the Qwen3.5-family admission path on both gfx1100 and gfx1201: battery and chain pass with coherent decoded text. Load behavior is also exercised. However, the behavioral change at crates/hipfire-runtime/src/llama.rs:1917 is untested because no plain-Llama/Qwen3 V2 fixture ran, and the central discriminating-kernel claim is unsupported because hw-gate.json contains no mq4v2_gemm_parity output or swapped-header negative-control results. The generic Redline pass uses qwen3.6-27b.mq4 and cannot substitute for direct MQ4V2 parity. Human review is therefore required before accepting the uncovered fallback and parity claims.
…llama and qwen35; discriminating GEMM parity; spec §9) to staging
There was a problem hiding this comment.
Both lanes pass and I closed the three gaps Sol flagged. A base-vs-PR Redline A/B on qwen3.8:27b-mq4-xt (gfx1201) is bit-identical — same prefill/decode launch sequences and the same logits/KV/recurrent hashes — so the qwen35 delegation is a true no-op, and HIPFIRE_MQV2_GFX11_WMMA=0 correctly leaves gfx12 alone. I ran the PR's mq4v2_gemm_parity on gfx1201: the disjoint-halves arm passes (v2 rel-rms ~2-3e-4) with the swapped-header control disagreeing by 0.5-1.6, so the negative control is doing real work. The plain-Llama refusal can't be exercised on hardware because no arch 0/1 MQ-V2 artifact exists in the registry, but I traced forward_prefill_chunk myself (llama.rs:2577-2605, :2731-2745) and agree master's gfx12 admission skipped the rotate and ran V1 launchers on V2 blobs — the refusal fixes a silent-corruption hole rather than removing working behavior. One small follow-up when convenient: the new comment in gemm_mq4g256v2_residual_mmq.hip cites gemm.rs:17860-17909 for the batch>=128 gate, but that is the launcher helper; the actual gate is at the WMMA wrapper call sites (e.g. gemm.rs:28029-28036) and also excludes replay-recording and graph-capture. Merging to staging. The PR has exactly one behavior change and one refactor. The refactor — qwen35::is_batchable_la (prefill.rs:1543-1560) delegating to llama::mqv2_wmma_batchable (llama.rs:1871-1880) — is proven a no-op on gfx1201 by a base-vs-PR Redline A/B on the real MQ4G256V2 fixture qwen3.8:27b-mq4-xt (sha256 9f91556f…, size 14980361216): prefill-128/512 sequence hashes, launch counts (1140/2280, i.e. two 256-token chunks of batched WMMA prefill), decode sequence hash, and logits/KV/recurrent output hashes are identical between $HW_GATE_BASE_BIN/daemon and $HW_GATE_BIN/daemon ($HW_GATE_EVIDENCE/ab-summary-qwen38-mq4xt-gfx1201.json), and HIPFIRE_MQV2_GFX11_WMMA=0 leaves gfx12 unchanged. The mandatory lanes add coherent battery+chain output for the same fixture on gfx1100 and gfx1201. The behavior change — llama::is_batchable_la (llama.rs:1917-1947) refusing MQ4G256V2/MQ4CG256/MQ6-2G256V2 on every arch instead of admitting them on gfx12 — only affects arch_id 0/1 (carriers.rs:733-737), for which the registry carries no MQ-V2 artifact (qwen3:0.6b/8b are .hf4). I verified the author's justification directly: forward_prefill_chunk's matchers (llama.rs:2577-2580, :3032, :3124, :3255) list only V1 dtypes, so an admitted V2 layer skips the FWHT rotate and lands on gemm_qkv_hfq4g256 / gemm_hfq4g256_residual / gemm_gate_up_hfq4g256 — master's gfx12 path for that class was silently wrong, and the PR moves it to per-token decode. That is a fail-closed rule that refuses no real artifact and replaces incorrect behavior, so it is not a regression under the master-is-the-oracle standard. The new mq4v2_gemm_parity disjoint-halves arm was run from the PR build on gfx1201 and passes with a negative control that separates correct and swapped-header decodes by three to four orders of magnitude ($HW_GATE_EVIDENCE/mq4v2_gemm_parity-gfx1201-PR.log). The two .hip changes are comment-only and the daemon is untouched. One doc nit for the author: gemm_mq4g256v2_residual_mmq.hip:12-14 cites gemm.rs:17860-17909 for the batch>=128 && batch%128==0 selection, but that range is the launcher helper; the gate lives at the WMMA wrapper call sites (e.g. gemm.rs:28029-28036) and also requires not-recording and not-capturing. To reverse this decision a maintainer would need either an arch 0/1 MQ-V2 artifact that decodes coherently through master's batched path on gfx12 (the source says it cannot), or a gfx11 A/B showing the qwen35 sequence hash changed. Override Sol needs-human: I closed Sol's three gaps on this host. (1) The added mq4v2_gemm_parity disjoint-halves arm was run on gfx1201 from the PR build and PASSES with a load-bearing negative control (v2 rel-rms 1.8e-4..3.1e-4 vs swapped-header reference 0.53..1.62 across batch 1/8/12/16/32). (2) The qwen35 admission change is proven a no-op by a base-vs-PR Redline A/B on the real MQ4G256V2 artifact qwen3.8:27b-mq4-xt on gfx1201: prefill-128/512 and decode launch sequences, launch counts, and logits/KV/recurrent output hashes are bit-identical, and HIPFIRE_MQV2_GFX11_WMMA=0 leaves gfx12 untouched as specified. (3) The plain-Llama V2 refusal cannot be exercised on any host because no arch 0/1 MQ-V2 artifact exists in the registry (the only arch-1 entries are qwen3:0.6b/8b .hf4); but the source trace is unambiguous that master's gfx12 admission ran the V1 hfq4g256 launchers on V2 blobs with the FWHT rotate skipped (llama.rs:2577-2605, :2731-2745), so the PR converts a silently-incorrect path into per-token decode and no working behavior is lost. A 'perf regression' measurement against a path that produced wrong activations is not meaningful.
First kernel-bucket run (warpfront#690, 33892920406): both lanes passed, then both seats returned empty text. omp log: "Pre-prompt context maintenance triggered by pending prompt size" contextTokens=1,357,577 against a 1,000,000 window. The Redline report's decode.captures (516 KB of raw per-dispatch records per lane) was inlined verbatim into the verdict and decide prompts with the rest of hw-gate.json. run.py: elide_captures() replaces capture-dump lists/dicts over 4 KB (keys captures/dispatches/packets/raw) with their entry count; every verdict field (pass, sequence_stable, measurement, aql_shadow, failures) stays, and the full report is still on disk as hw-gate-logs/redline.json in the artifact. On warpfront#690's real evidence: 1,282,896 -> 41,435 bytes per lane; seat prompt ~10 K tokens. review.py: evidence_for_prompt() caps inlined evidence at 600 KB with an explicit truncation marker (the diff was already capped at 400 KB; the evidence was not) so no future field can push a seat past the window. 103/103 in scripts/hw-gate/tests.
Every rung of the 2026-09-04 ladder hit the same 409 on the staging merge: `crates/*/map.md` carries a `<!-- crate-map:generated -->` block that both branches regenerate, so any two PRs touching the same crate conflict there while their real code merges cleanly. warpfront#689, warpfront#690, warpfront#691, warpfront#686, warpfront#687, warpfront#688 and warpfront#682 all needed the same three manual steps -- merge staging in, regenerate the block with scripts/check-crate-maps.py, merge -- six of them tonight. A gate that decides merge-staging and then holds on a generated file is asking a human to run a script, which is not review. On a 409 the decide phase now retries locally: merge staging into the PR head, and if the conflicted set is generated maps only, re-run check-crate-maps.py for those crates, commit, and merge the result. The retry is deliberately narrow, because auto-resolving conflicts is exactly where a gate can do damage: - if ANY conflicted path is not a `map.md`, it declines and the hold stands with the offending paths named -- a real code conflict must reach a human - it regenerates rather than picking a side, so the committed block is what the tree actually generates, not whichever branch won - a failed regeneration, a git error, or a timeout all decline rather than force Test: `test_generated_map_retry_refuses_real_code_conflicts` builds a real repo with a conflicting `.rs` and asserts the retry returns no merge SHA and names the file. The guard is the part worth pinning; the happy path is exercised by the ladder itself. 122/122 hw-gate tests pass.
Summary
Follow-ups from the MQ4G256V2 kernel-family audit (
docs/plans/audit-2026-09-02/audit-Mq4v2Kernels.md, PR #685):hipfire_runtime::llama::is_batchable_laadmitted MQ-V2 (qt 44/47–50) WMMA prefill only on gfx12;qwen35::is_batchable_laadmitted gfx11 + gfx12 with theHIPFIRE_MQV2_GFX11_WMMAkill-switch. Both doc-comments claimed they "match exactly". The rule now lives once inllama::mqv2_wmma_batchable/llama::mqv2_gfx11_wmma_enabled_from_env; both gates delegate; a test asserts agreement over 6 dtypes × {gfx1100, gfx1151, gfx1201, gfx1030, gfx1010}.MQ4CG256(qt=45) stays gfx12-only in both, per the spec. Behaviour change (revised in4e360fb2a): plain Llama/Qwen3 dense models on MQ-V2 stay per-token on every arch —llama.rs::forward_prefill_chunkhas no V2 arms (matchers at ~2570/3025/3117/3248), so admitting them would run V1 launchers on V2 blobs; the hw-gate Fable seat caught this by source trace (run 33895641944). The shared rule still admits qwen35 on gfx11+gfx12 (its chunk path has the V2 arms). master's old gfx12-only arm had the same hole for llama; this PR closes it instead of widening it.mq4v2_gemm_parityused Gaussian weights whose two halves have near-identical headers, so a wrong half-select hides inside quantization noise — contradicting its own header comment. It gains a disjoint-halves arm (half 0 in[-1,1], half 1 in[96,160], same batch sweep) with a negative control (swapped-header reference must disagree) and a CPU test that the fixture discriminates.kt += 2),gemm_mq4g256v2_residual_mmq.hip"Experimental" → production fast path (gfx1100/1151, batch ≥128),gemm_qkv_mq4g256v2_wmma.gfx12.hip"HYPOTHESIS" → the validated C-map statement its sibling carries.Which crate(s) does this touch?
crates/hipfire-runtime,crates/hipfire-arch-qwen35,kernels/(comments only), docsEvidence
mq4v2_residual_parity— PASS on gfx1100 (hipx) and gfx1201 (hiptrx): all shapes, swapped-headers baseline at 0.9996 rel error.mq4v2_gemm_parity— PASS on gfx1201 (the example calls the*_gfx12launchers directly; gfx12-only by construction): disjoint arm v2 rel-rms 2.5e-4 vs 1.263 with swapped headers; gate_up/qkvza v1/v2 arms unchanged.qwen3-8b.hf4is qt=6). If a reviewer wants that before merge, it needs a--format mq4quantization of qwen3-8b first.Test plan
cargo test -p hipfire-runtime -p hipfire-arch-qwen35— 597 + 189 pass, incl.mqv2_admit_llama_qwen35_lockstepanddisjoint_fixture_discriminates_half_select