Skip to content

server: add KV cache quantization flags (--kv-bits, --kv-group-size, --quantized-kv-start)#1353

Open
soobrosa wants to merge 4 commits into
ml-explore:mainfrom
soobrosa:feat/server-kv-cache-quant
Open

server: add KV cache quantization flags (--kv-bits, --kv-group-size, --quantized-kv-start)#1353
soobrosa wants to merge 4 commits into
ml-explore:mainfrom
soobrosa:feat/server-kv-cache-quant

Commits

Commits on Jun 7, 2026

Commits on Jun 11, 2026

Commits on Jun 14, 2026