vllm-serving

Here are 2 public repositories matching this topic...

The Continuous Verification and Optimization layer for self-hosted LLMs

self-hosted mlops vllm llm-inference vllm-serving

A simple UI and config generator to run vLLM with Docker, GPU settings, model config parsing, memory estimation, and OpenAI-compatible test clients.

python docker flask docker-compose cuda flask-application model-serving mlops huggingface llm vllm vllm-ui vllm-serve vllm-server vllm-serving

To associate your repository with the vllm-serving topic, visit your repo's landing page and select "manage topics."