feat(BACKEND-TENSTORRENT-KEEPQUANT): Q5_K/Q6_K/Q8_0 decode chains and the q4km vehicle gate (#2959)

.agents/backend-matrix.md

@@ -246,7 +246,7 @@ this repository. State remains `ACTIVE`; no lifecycle transition is claimed.
 | `BACKEND-TENSTORRENT` | Tenstorrent Blackhole (Tensix multicore, discrete PCIe, no unified memory) — thin `vt::` adapter over ttnn's existing C++ op library rather than hand-written kernels, mirroring the Metal/MLX decision (E1); vLLM has no Tenstorrent platform anywhere | vllm.cpp extension through upstream seam `platforms/interface.py:134-229` (same pattern as Metal/Vulkan) | **ACTIVE 2026-08-10.** `vt::tenstorrent::Backend` + registrar [tenstorrent_backend.cpp](../src/vt/tenstorrent/tenstorrent_backend.cpp); shared mesh-device lifecycle [tenstorrent_device.cpp](../src/vt/tenstorrent/tenstorrent_device.cpp); 17 registered ops cover OPT-125m and the Qwen3-0.6B forward (`kMatmul`, `kMatmulBT`, `kAdd`, `kRelu`, `kEmbedding`, `kLayerNorm`, `kRmsNorm`, `kSiluAndMul`, bf16/f32 casts, three RoPE forms, `kQkvSplit`, `kReshapeAndCache`, host-oracle `kPagedAttention`, `kGreedyArgmax`) [tenstorrent_ops.cpp](../src/vt/tenstorrent/tenstorrent_ops.cpp); platform allow-list selects OPT and Qwen3 [platforms/tenstorrent.cpp](../src/vllm/platforms/tenstorrent.cpp). `DeviceType::kTENSTORRENT` [device.h](../include/vt/device.h) | [test_tenstorrent_backend.cpp](../tests/vt/test_tenstorrent_backend.cpp) carries real-Blackhole op gates; [test_qwen3_paged_engine.cpp](../tests/parity/test_qwen3_paged_engine.cpp) selects Tenstorrent device-specific anchor and teacher-forced near-tie goldens. OPT-125m STRICT 6/6 passed. Qwen3 short warm smoke ran 4 tokens at about 0.28 tok/s; full 16x16 gate remains pending behind host paged attention | [tenstorrent-backend.md](specs/tenstorrent-backend.md) | `ACTIVE` | `CLAIM-BACKEND-TENSTORRENT-SPIKE` |
 | `BACKEND-TENSTORRENT-RESIDUAL-GOLDEN` | Child of `BACKEND-TENSTORRENT` — the owed op-level numerics evidence at the residual-RMS device boundary (`kDeviceResidualMinRows == 32`): device path does `ttnn::add`+`ttnn::rms_norm` in bf16; host/CPU path accumulates in f32. Bot-flagged on #289; never measured at the boundary. | vllm.cpp CPU oracle `RmsNormKernel` mirrors vLLM `fused_add_rms_norm` (add in model dtype, variance in f32); `src/vt/cpu/cpu_ops.cpp:371-398` | `src/vt/tenstorrent/tenstorrent_ops.cpp:1067-1117` (host/device split, `kDeviceResidualMinRows=32`) | [test_tenstorrent_backend.cpp](../tests/vt/test_tenstorrent_backend.cpp) `kRmsNorm residual: device vs CPU f32 oracle across the rows=32 boundary`: 22/22 cases on real Blackhole P150. **Measured 2026-08-11:** host path `rows<32` bit-identical to CPU (`max_abs=0`); device bf16 path `rows>=32` diverges by constant **0.0459 abs** (1.9–2.6× rel on near-zero outputs) — bf16 rounding signature, not accumulation. Decision pending the e2e golden tie-break | [tenstorrent-residual-golden.md](specs/tenstorrent-residual-golden.md) | `SPIKE` | `CLAIM-BACKEND-TENSTORRENT-RESIDUAL-GOLDEN` |
 | `BACKEND-TENSTORRENT-MISTRAL` | Child of `BACKEND-TENSTORRENT` — allowlist `MistralForCausalLM` (Mistral-7B-v0.3: GQA 32/8, head_dim 128, plain rope theta 1e6, untied lm_head, full attention) on the TT platform + device-aware SACRED gate. Mistral reuses the Qwen3-dense forward verbatim (qk-norm skipped); every op already registered. No new kernel. | vLLM `mistral.py::MistralForCausalLM(LlamaForCausalLM)` (already ported to the shared dense machinery); gate pattern mirrored from `test_qwen3_paged_engine.cpp:221-296` | `src/vllm/platforms/tenstorrent.cpp:52-54` (allowlist) + `tests/parity/test_mistral_paged_engine.cpp` (device-aware wiring + Backend Proof) | **Gate PASSED on Blackhole P150 (2026-08-12):** [test_mistral_paged_engine.cpp](../tests/parity/test_mistral_paged_engine.cpp) 16/16 prompts PASS (12/16 strict-exact, 4/16 near-tie, 0 forward-divergent), max gap **0.062 nats**, BACKEND PROOF 0 declines (kMatmul selections=256 = untied lm_head on device, kPagedAttention=8192). Goldens `our_ids_tenstorrent.npy` + `neartie_gap_mnats_tenstorrent.npy` (transformers alternative-oracle; POL-ORACLE deviation recorded, same as Qwen3-0.6B TT precedent). Exit SIGSEGV 139 is the known MeshDevice teardown crash, not a gate failure | [tenstorrent-mistral.md](specs/tenstorrent-mistral.md) | `ACTIVE` | `CLAIM-BACKEND-TENSTORRENT-MISTRAL` |
-| `BACKEND-TENSTORRENT-GDN` | Child of `BACKEND-TENSTORRENT` — the GDN linear-attention op chain as native TT kernels, the hard prerequisite for the Qwen3.5/3.8 family (#1715): `kGdnPrefill`, `kGdnDecode`, `kL2Norm`, `kRmsNormGated`, `kCausalConv1dFwd`/`kCausalConv1dUpdate`, `kGdnStateGather`/`kGdnStateScatter`. The P150 is discrete, so an op miss refuses by name — the ops must land before any `Qwen3_5*` arch registration. Correctness oracle is our own CPU f32 arm (residual-golden precedent); no vLLM mirror exists for TT | Substrate: pinned tt-metal `ttnn::transformer::chunk_gated_delta_rule` (FLA chunked GDN forward, on-core recurrent state, `initial_state`/`final_state`) behind a varlen+state-permute adapter for `kGdnPrefill`; decode = rank-1 update composed from ttnn matmul+eltwise with a device shadow keyed by host pointer (`PagedKvShadow` pattern); contracts at `src/vt/ops.cpp:1823-2500`, CPU reference `src/vt/cpu/cpu_ops.cpp:1537-1740` | GDN kernels [GdnPrefillKernel :4220](../src/vt/tenstorrent/tenstorrent_ops.cpp#L4220) + [GdnDecodeKernel :5049](../src/vt/tenstorrent/tenstorrent_ops.cpp#L5049), registered [tenstorrent_ops.cpp:6755-6770](../src/vt/tenstorrent/tenstorrent_ops.cpp#L6755-L6770) (`kL2Norm`..`kGdnStateScatter`); op-level cases vs the CPU f32 oracle (T-sweep, indexed-`state_idx` forms, prefill↔decode state round-trip) | [test_tenstorrent_backend.cpp:1749-3340](../tests/vt/test_tenstorrent_backend.cpp#L1749-L3340): L2Norm, RmsNormGated, CausalConv1dFwd/Update, GdnPrefill, GdnDecode, prefill↔decode round-trip, StateGather/Scatter, edge shapes — every op family vs the CPU f32 oracle. W1 `34fde3502` (prefill set) + W2 `c85af0aaf` (decode+state-I/O set) landed, both fresh-review PASS; production-reached via the `Qwen3_5*` wiring row (BACKEND-TENSTORRENT-QWEN35: allow-list, e2e sacred pair 16/16 STRICT both legs); capture compatibility unmeasured behind #1625; closure [parity-ledger.md#L946](parity-ledger.md#L946) | [tenstorrent-gdn.md](specs/tenstorrent-gdn.md) | `DONE` | `c85af0aaf` |
+| `BACKEND-TENSTORRENT-GDN` | Child of `BACKEND-TENSTORRENT` — the GDN linear-attention op chain as native TT kernels, the hard prerequisite for the Qwen3.5/3.8 family (#1715): `kGdnPrefill`, `kGdnDecode`, `kL2Norm`, `kRmsNormGated`, `kCausalConv1dFwd`/`kCausalConv1dUpdate`, `kGdnStateGather`/`kGdnStateScatter`. The P150 is discrete, so an op miss refuses by name — the ops must land before any `Qwen3_5*` arch registration. Correctness oracle is our own CPU f32 arm (residual-golden precedent); no vLLM mirror exists for TT | Substrate: pinned tt-metal `ttnn::transformer::chunk_gated_delta_rule` (FLA chunked GDN forward, on-core recurrent state, `initial_state`/`final_state`) behind a varlen+state-permute adapter for `kGdnPrefill`; decode = rank-1 update composed from ttnn matmul+eltwise with a device shadow keyed by host pointer (`PagedKvShadow` pattern); contracts at `src/vt/ops.cpp:1823-2500`, CPU reference `src/vt/cpu/cpu_ops.cpp:1537-1740` | GDN kernels [GdnPrefillKernel :4220](../src/vt/tenstorrent/tenstorrent_ops.cpp#L4220) + [GdnDecodeKernel :5049](../src/vt/tenstorrent/tenstorrent_ops.cpp#L5049), registered [tenstorrent_ops.cpp:7244-7260](../src/vt/tenstorrent/tenstorrent_ops.cpp#L7244-L7260) (`kL2Norm`..`kGdnStateScatter`); op-level cases vs the CPU f32 oracle (T-sweep, indexed-`state_idx` forms, prefill↔decode state round-trip) | [test_tenstorrent_backend.cpp:1749-3340](../tests/vt/test_tenstorrent_backend.cpp#L1749-L3340): L2Norm, RmsNormGated, CausalConv1dFwd/Update, GdnPrefill, GdnDecode, prefill↔decode round-trip, StateGather/Scatter, edge shapes — every op family vs the CPU f32 oracle. W1 `34fde3502` (prefill set) + W2 `c85af0aaf` (decode+state-I/O set) landed, both fresh-review PASS; production-reached via the `Qwen3_5*` wiring row (BACKEND-TENSTORRENT-QWEN35: allow-list, e2e sacred pair 16/16 STRICT both legs); capture compatibility unmeasured behind #1625; closure [parity-ledger.md#L946](parity-ledger.md#L946) | [tenstorrent-gdn.md](specs/tenstorrent-gdn.md) | `DONE` | `c85af0aaf` |
 | `BACKEND-TENSTORRENT-GDN-DEVICE-PURE` | Child of `BACKEND-TENSTORRENT-GDN` — make the decode-side GDN ops (`CausalConv1dUpdateKernel`, `GdnDecodeKernel`) device-resident so a tt-metal trace capture admits them, unblocking the Qwen3.5-0.8B captured arm, the last capture-blocked family with a committed eager pair (#2907, owed from #2812) | Move the per-call host orchestration on-device: resident inputs from the graph's producer ops, device-side indexed state update (baked slot addressing re-primed by the recapture cadence, or `kGdnStateGather`/`kGdnStateScatter` indirection — decided on recapture-cost measurement), conv two-views coherent on device, token readback outside the captured span | [CausalConv1dUpdateKernel :5013](../src/vt/tenstorrent/tenstorrent_ops.cpp#L5013), [GdnDecodeKernel :5225](../src/vt/tenstorrent/tenstorrent_ops.cpp#L5225) (EnsureHost ×5, ReadIdxHost, UploadTensor ×5+), q35 harness pair selection [test_qwen35_paged_engine.cpp:233](../tests/parity/test_qwen35_paged_engine.cpp#L233), fatal repro `fd_mesh_command_queue.cpp:760` 2/2 | Red-first: the #2812 loud-skip opt-in cells run captured; captured dump ×2 byte-identity with reset between; teacher-forced pair vs the transformers oracle inside the eager band; Qwen3.5 joins `DecodeCaptureDefaultArch`, ambient adjudicates CAPTURED, env=0 eager; tamper + arch-deletion mutations red; gate arm selection [test_qwen35_paged_engine.cpp:301-330](../tests/parity/test_qwen35_paged_engine.cpp#L301-L330); closure [parity-ledger.md:947](parity-ledger.md#L947) | [tenstorrent-gdn-device-pure.md](specs/tenstorrent-gdn-device-pure.md) | `DONE` | `bcade48d6` |
 | `BACKEND-TENSTORRENT-KEEPQUANT` | Child of `BACKEND-TENSTORRENT` — keep-quant dense dot on TT-Metal ([#2959](https://github.com/mudler/vllm.cpp/issues/2959)): GGUF k-quant arms refuse on the P150 until this row lands — W1 (#2989, open) lands the Q4_K decode, W2 (branch) lands the dot and the `kTENSTORRENT` predicate arm (`gguf_keep_quant.cpp:136-148`) admitting exactly `{Q4_K}` — while the smallest Qwen3.8 artifact that fits the card is the dense 27B Q4_K_M (17.1 GB; bf16 is 53.8 GB) and a quantized Qwen3.5 (0.8B, ~0.5 GB) is the test vehicle | ggml k-quant block formats via our reader, pinned bit-exact vs llama.cpp `b10451` (#2240/#2260 lineage); no vLLM mirror for TT (secondary substrate, deviation by design); substrate surveyed 2026-09-05: tt-metal has NO packed-weight matmul (`ttnn` quantization is per-tensor activation dquant only), so the kernel is ours — resident blocks, on-core decode to bf16 tiles through f32 (decode bit-exact vs `BlockToFloat`), the dot reusing the `kMatmulBT` tile path, device-bf16 band at the residual-golden boundary, the int8 dot as the named next lever | W1 LANDED (#2989): the Q4_K block-decode device path. W2 ON BRANCH: the dot (`MatmulBTQuantKernel`, decode → one bf16 RNE → `kMatmulBT` tile matmul, reached through `vt::MatmulBT`'s block-weight dispatch) + the predicate arm admitting exactly the registered set (never wider — the GLM-5.3 W10 lesson). OWED W3: capture-safe residency + the e2e vehicle. OWED W4: Q5_K/Q6_K/Q8_0, the int8 lever, the 27B arm | W1 LANDED (#2989): decode bit-exact vs `vt::cpu::BlockToFloat` across a shape sweep (red-first, op-level suite pattern). W2 ON BRANCH: the dot vs the decode-based bf16 oracle inside the analytic operand-rounding envelope + the predicate routing set (both red-first). OWED W3: capture dump ×2 byte-identity (#2907 discipline); e2e 16-prompt battery on the P150 vs the bf16 arm's committed pair (STRICT or inside the ≤500-mnat near-tie band) | [tenstorrent-keepquant.md](specs/tenstorrent-keepquant.md) | `ACTIVE` | `CLAIM-BACKEND-TENSTORRENT-KEEPQUANT` — implementer on the row branch; spec commit on `row/BACKEND-TENSTORRENT-KEEPQUANT`, claim in [.agents/claims/](claims/CLAIM-BACKEND-TENSTORRENT-KEEPQUANT.md) |
 | `BACKEND-TENSTORRENT-QWEN35` | Child of `BACKEND-TENSTORRENT` — the wiring row: `Qwen3_5ForConditionalGeneration` (dense text GDN hybrid) on the TT allow-list, the op delta the family forward refuses by name (`kGdnPostConv`, `kSigmoidGateBf16`, `kAttnQkNormRopeGate`, `kAttnQkNormRope` — pinned empirically by a W0 refusal sweep), and the first e2e gate. Makes the GDN row's ops production-reached (#1715 stays open until the family runs; GDN row lifecycle moves in the same change this lands) | Substrate: the GDN row's 8 kernels + the 27-op TT registry; e2e mirrors the ratified Mistral TT golden treatment (`VT_DUMP_IDS` bootstrap → `scripts/qwen3-neartie-gap-transformers.py` teacher-forced near-tie gaps → committed device-golden pair, POL-ORACLE deviation); op-level oracle is the CPU f32 arm. Capacity: `Qwen/Qwen3.5-0.8B` bf16 ≈1.6 GB fits (proven envelope Mistral-7B ≈14.5 GB); 27B bf16 ≈53.8 GB and GGUF k-quant arms (no TT kernels) refused by name, owed | `src/vllm/platforms/tenstorrent.cpp` (allow-list, lands last) + `src/vt/tenstorrent/tenstorrent_ops.cpp` (op delta) + `tests/parity/test_qwen35_paged_engine.cpp` (TT arm) | Spec committed 2026-08-23; W0 sweep, W1 op delta, W2 e2e, W3 GDN-reviewer leftovers (d2h counter completeness, `conv_transposed` fast-path check) owed | [tenstorrent-qwen35.md](specs/tenstorrent-qwen35.md) | `ACTIVE` | `CLAIM-BACKEND-TENSTORRENT-QWEN35` |