bench(dsv4): Stage 0.5 mini-harness — pure-PyTorch DSV4-Flash KV port + KakeyaLattice probe

[FluffyAIcode]

Scope

Smallest honest experiment addressing the question flagged in the DeepSeek-V4-Pro / V4-Flash discussion: does KakeyaLattice's five-lever + D4/E8 shaping-gain stack still buy anything on V4-architecture KV cache?

Status: scaffold + H200 results landed. Draft — not to be auto-merged per user instruction.

H200 results (2026-04-24)

Host: google/gemma-4-E4B post-embedding hidden states (projected 2560 → 4096)
Hardware: NVIDIA H200, torch 2.11.0+cu130, transformers 5.5.2, native fp8_e4m3fn
Input: 1 × 2048-token WikiText-style passage

Headline

$E_8$ Q=38 beats FP8 per-64-block baseline on ALL THREE V4 KV streams at 78% of the bits.

stream	FP8 bits	FP8 rel-MSE	$E_8$ Q=38 bits	$E_8$ Q=38 rel-MSE	$E_8$/FP8	bit savings
sliding_window_kv	4224	$7.27\times10^{-4}$	3296	$\mathbf{6.17\times10^{-4}}$	$\mathbf{0.849\times}$	$-22%$
csa_pool_kv_ratio4	4224	$9.03\times10^{-4}$	3296	$\mathbf{7.84\times10^{-4}}$	$\mathbf{0.868\times}$	$-22%$
hca_pool_kv_ratio128	4224	$1.12\times10^{-3}$	3296	$\mathbf{9.15\times10^{-4}}$	$\mathbf{0.820\times}$	$-22%$

First empirical evidence that KakeyaLattice has a Pareto advantage over V4-Flash's internal FP8 quantisation on V4-architecture KV.

Non-Gaussian audit: all four paper gates fire on all three streams

stream	|kurt-3| (gate 0.5)	iso-var (gate 1.5)	Had-var (gate 1.5)	RMS W2/σ (gate 0.05)
sliding_window_kv	0.95	15.9	11.9	0.244
csa_pool_kv_ratio4	0.99	22.3	22.7	0.350
hca_pool_kv_ratio128	1.10	2515	231	0.470

Reference Qwen3-4B (paper §1.3): kurt=0.84, iso=4.71, W2/σ=0.65. V4-arch KV is at least as non-Gaussian as Qwen3-4B, and 3–500× more anisotropic. The five engineering levers are fully motivated on V4 KV.

$E_8$ universal win over $D_4$ at matched Q

stream	$D_4$ Q=38	$E_8$ Q=38	$E_8/D_4$	dB gain
sliding_window_kv	9.34e-04	6.17e-04	0.661	+1.80
csa_pool_kv_ratio4	1.18e-03	7.84e-04	0.665	+1.77
hca_pool_kv_ratio128	1.37e-03	9.15e-04	0.668	+1.75

Mean $E_8/D_4$ ratio $0.665\times$ (+1.78 dB) matches the paper's Qwen3-4B measurement (+1.87 dB) to within noise. The E8 shaping-gain + super-linear amplification pattern transfers cleanly to V4-arch KV.

What's in the PR

Code (~1000 lines)

File	Purpose
benchmarks/dsv4_stage0_5/dsv4_kv_generator.py	Pure-PyTorch port of DSV4-Flash inference/model.py (Compressor + Main KV + RoPE + FP8 sim)
benchmarks/dsv4_stage0_5/test_dsv4_generator.py	8 unit tests (CPU + GPU, all passing)
benchmarks/dsv4_stage0_5/run_dsv4_synthetic.py	CPU-friendly synthetic smoke + CI frozen reference generator
benchmarks/dsv4_stage0_5/run_dsv4_stage0_5.py	Rigorous H200 harness with real host-model hidden states

Results + docs

File	Purpose
reports/v1_5_release/dsv4_stage0_5/README.md	Experiment description + six honesty caveats
reports/v1_5_release/dsv4_stage0_5/FINDINGS.md	Full H200 findings with reproducibility
reports/v1_5_release/dsv4_stage0_5/dsv4_stage0_5_gemma4_e4b.json	H200 raw output
reports/v1_5_release/dsv4_stage0_5/dsv4_stage0_5_synthetic_reference.json	CI frozen reference

Port fidelity

Operator-level port from deepseek-ai/DeepSeek-V4-Flash/inference/model.py commit 6e76323:

V4-Flash reference	Port
Compressor.forward (prefill + overlap-transform)	DSV4Compressor
Attention.forward wkv + kv_norm + RoPE + FP8 sub-path	DSV4MainKVProjection
precompute_freqs_cis + apply_rotary_emb	verbatim
RMSNorm	verbatim
kernel.act_quant (FP8 in-place)	_simulate_fp8_block_quant_dequant (uses native float8_e4m3fn on CUDA, fake-quant fallback on CPU)

Not ported: sparse_attn_kernel (attention is out of scope for Stage 0.5); hc_split_sinkhorn (HC bypassed, documented caveat).

Honesty caveats (unchanged, see README.md)

1. Weights are random Gaussian-init, not V4-trained.
2. Three layers captured, not all 43.
3. No Indexer (side path producing indices, not KV values).
4. No Hyper-Connections (bypassed; learned linear rebalancing expected to soften kurtosis somewhat but not flip the sign).
5. Single passage, n=2048 tokens.
6. No Δppl (requires full 43-layer stack + trained weights + MoE, out of scope).

Next steps (unchanged plan)

• Do NOT merge this PR per user instruction — keeps the scaffold reviewable.
• Stage 1: once vLLM lands DeepseekV4Attention, run rigorous_eval.py on trained V4-Flash weights (2–4× H200 NVL node) to validate the Δppl story end-to-end.

What this PR implies for the paper

The Stage 0.5 H200 result — $E_8$ Q=38 wins $-22%$ bits and $-15$ to $-18%$ K-MSE over FP8 per-64 on all three V4 streams — is concrete enough that the paper's "Conclusion" section can add a one-sentence forward reference:

"Stage 0.5 architectural probe (reports/v1_5_release/dsv4_stage0_5/) on a pure-PyTorch port of DeepSeek-V4-Flash's KV write-path shows that the $E_8$ variant retains a $0.82$–$0.87\times$ rel-MSE advantage over FP8 per-64-block at 78% of the bits, suggesting that the shaping-gain machinery transfers to V4's hybrid CSA/HCA attention architecture."

But we're NOT adding this to the paper in this PR — this PR is scaffold + first-run-on-H200 only.