| Documentation | Blog | Paper | Twitter/X | User Forum | Developer Slack |
🔥 We have built a vllm website to help you get started with vllm. Please visit vllm.ai to learn more. For events, please visit vllm.ai/events to join us.
This is RunKV version of vLLM(Forked From vanilla vLLM).
| Feature | Code | Test |
|---|---|---|
| Decoupled-Paged Attention | ✅ | ✅ |
| UVA-based Copy | ✅ | ✅ |
| Compute-IO Overlapping | ✅ | ✅ |
| IO & Recompute Policy | 🚧 | 🚧 |
| Reservation & Eviction Policy | 🚧 | 🚧 |
| Dynamic Buffers' Size | 🚧 | 🚧 |
| Dynamic Buffers' Layout | 🚧 | 🚧 |
