Self-hosted llama.cpp serving stack for Muse-Glimmer-30B GGUF with DFlash2 speculative decoding on Kaggle NVIDIA T4x2 and an authenticated OpenAI-compatible API gateway.
cuda self-hosted kaggle multi-gpu llama-cpp llm-inference gguf speculative-decoding openai-compatible nvidia-t4 dflash2 muse-glimmer-30b
-
Updated
Sep 6, 2026 - Python