feat(kvcache): store the KV cache as fp8 e4m3 codes (--kv-cache-dtype… - #354

Open

ArqAlice wants to merge 13 commits into

FlashML-org:mainfrom

ArqAlice:feat/fp8-quantization

Commits on Sep 2, 2026

feat(kvcache): store the KV cache as fp8 e4m3 codes (--kv-cache-dtype fp8)
ArqAlice
committed
03fb043

Commits on Sep 4, 2026

fix(kernels): give the V tensor its own row pitch in the fp8 KV store

MT-z
and
claude
committed
811ccee
Merge pull request #1 from MT-z/fix/kv-fp8-vstore-pitch
ArqAlice
authored
5b9efc5
Merge branch 'FlashML-org:main' into feat/fp8-quantization
ArqAlice
authored
3e5bbdd

Commits on Sep 5, 2026

Commits on Sep 6, 2026

perf(kernel): apply the fp8 KV dequant scale after the dot, not to the tile

naerymdan
and
claude
committed
05861fb
perf(kernel): size the extend tile from the KV cache element size

naerymdan
and
claude
committed
73ca76c

Commits on Sep 7, 2026