Analytical benchmark connecting tensor parallelism and KV cache management in LLM serving: GQA sharding regimes, all-gather vs recompute tradeoffs, PCIe viability, and operational decision matrix across NVLink, InfiniBand, and PCIe
performance-engineering benchmark simulation inference systems attention infiniband disaggregation pcie nvlink analytical-model gqa kv-cache tensor-parallelism llm llm-serving grouped-query-attention machine-learning-infrastructure prefix-sharing all-gather
-
Updated
Jul 26, 2026 - Python