vLLM serving poolside Laguna S 2.1 (118B MoE, 8B active, INT4) on AMD Strix Halo (Ryzen AI Max+ 395, gfx1151, 128GB unified) via TheRock ROCm nightly. OpenAI-compatible, 256K context, DFlash speculative decoding.
rocm mixture-of-experts poolside openai-api vllm local-llm llm-inference speculative-decoding strix-halo gfx1151 int4-quantization ryzen-ai-max-395 laguna-s
-
Updated
Jul 23, 2026 - Python