Pinned Loading

1. TokenPowerBench TokenPowerBench Public

   chenxuniu/TokenSpark-Benchmark-Benchmarking-Power-Consumption-of-LLM-Inference-on-Multi-Node-Clusters

   Python 11

2. LLM-Inference-Engine-Benchmark LLM-Inference-Engine-Benchmark Public

   A comprehensive benchmarking tool for measuring energy consumption and power efficiency of Large Language Model (LLM) inference engines including vLLM, DeepSpeed, TensorRT-LLM, and Transformers. Fe…

   Python 5

3. awesome-disaggregated-llm-serving awesome-disaggregated-llm-serving Public

   A curated map of AFD, PD disaggregation, KV-cache systems, MoE serving, and re-aggregation baselines for LLM serving.

   Python

4. llm-power-profiler llm-power-profiler Public

   Lightweight local monitor for watts, tokens, and joules per token on OpenAI-compatible LLM servers.

   Python

5. vllm-project/semantic-router vllm-project/semantic-router Public

   System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge

   Go 4.2k 670