[AMD] vLLM Kimi MXFP4 & MiniMax M2.5 FP8 disaggregated prefill-decode for MI355X#1569

Merged

functionstackx merged 85 commits into

mainfrom

amd/vllm_disagg_mvp_dev2

May 27, 2026

Commits on May 27, 2026

[AMD] Add vLLM disaggregated prefill-decode benchmark for MI355X

chunfangamd
authored and
functionstackx
committed
[AMD] Refactor vLLM disagg recipe: models.yaml, UCX cleanup, QoS support

chunfangamd
authored and
functionstackx
committed
[AMD] Update vLLM disagg recipe for v0.17.1 NixlConnector API

chunfangamd
authored and
functionstackx
committed
[AMD] Make vLLM disagg recipe CI-compatible (mia1 cluster)

chunfangamd
authored and
functionstackx
committed
[AMD] Co-locate vLLM disagg router with prefill on NODE_RANK=0

chunfangamd
authored and
functionstackx
committed
[AMD] Use public vLLM base image with runtime dependency install

chunfangamd
authored and
functionstackx
committed
[AMD] Enable Expert Parallelism with MoRI all-to-all on vLLM disagg decode

chunfangamd
authored and
functionstackx
committed
[AMD] Switch vLLM disagg KV transfer to MoRI-IO with protocol-aware proxy

chunfangamd
authored and
functionstackx
committed
[AMD] BUG fix: RANDOM_RANGE_RATIO never reaches bench.sh

ichbinblau
authored and
functionstackx
committed
Bug fix: 1. With DRY_RUN=1, node 0 skipped starting proxy/prefill but still ran the first barrier; 2. kill and kill run only when DRY_RUN=0

ichbinblau
authored and
functionstackx
committed
[AMD] Fix vLLM disagg hang: READ mode support + safety timeouts

chunfangamd
authored and
functionstackx
committed
Adapt vLLM disagg recipe for 9N mia1 cluster (mlx5 NICs)

chunfangamd
authored and
functionstackx
committed
[AMD] Fix vLLM disagg sweep hang: KV cache leak + benchmark client hardening

chunfangamd
authored and
functionstackx
committed
[AMD] Fix vLLM disagg Slurm job never terminating after benchmark completion

chunfangamd
authored and
functionstackx
committed
[AMD] Enable MoRI-IO READ mode by default for vLLM disagg

chunfangamd
authored and
functionstackx
committed
[AMD] Fix CI checkout failure caused by root-owned __pycache__ files

chunfangamd
authored and
functionstackx
committed
[AMD] Fix CI checkout EACCES by redirecting Python bytecache off NFS

chunfangamd
authored and
functionstackx
committed
[AMD] Fix KV reaper deadlock on high-ISL disagg workloads

chunfangamd
authored and
functionstackx
committed
[AMD] Enable reading PREFILL_TP,PREFILL_EP,PREFILL_DP_ATTN,DECODE_TP,DECODE_EP,DECODE_DP_ATTN from amd-master.yaml config.

ichbinblau
authored and
functionstackx
committed
[AMD] Upgrade vLLM disagg image from v0.17.1 to v0.18.0

chunfangamd
authored and
functionstackx
committed
[AMD] Add Kimi-K2.5-MXFP4 disagg inference config (1P2D)

chunfangamd
authored and
functionstackx
committed
feat: add MiniMax M2.5 PD disaggregation recipe (1P2D, MoRI-EP + MoRI-IO)

authored and
functionstackx
committed
feat: add Dockerfile and runtime patch for MiniMax M2.5 WideEP + MoRI

authored and
functionstackx
committed
Fix: rename minimaxm25 to minimaxm2.5 for CI naming consistency

chunfangamd
authored and
functionstackx
committed
Optimize: add --gpu-memory-utilization 0.95 and --block-size 32 to MiniMax M2.5 disagg

chunfangamd
authored and
functionstackx
committed
Fix: MiniMax M2.5 disagg — require EP=8 for prefill, fix ROCm gate dtype

chunfangamd
authored and
functionstackx
committed
Remove unused docker/minimax-m25-disagg/ directory

chunfangamd
authored and
functionstackx
committed
remove vllm disagg for dpsr1 and dpv3

ichbinblau
authored and
functionstackx
committed
consolidate amd_utils for sglang and vllm

ichbinblau
authored and
functionstackx
committed
use vLLM router as default router for vllm disagg

ichbinblau
authored and
functionstackx
committed
fix bugs

ichbinblau
authored and
functionstackx
committed
[AMD] Bump to nightly vllm and vllm-router images (#1208)

simondanielsson
authored and
functionstackx
committed
update vllm image and vllm router image

ichbinblau
authored and
functionstackx
committed
update the interface prefix for tw cluster

ichbinblau
authored and
functionstackx
committed
add deps for ib device auto-detection

ichbinblau
authored and
functionstackx
committed
update vllm image

ichbinblau
authored and
functionstackx
committed
fix indentation and add missing finally block in async_request_openai_chat_completions

authored and
functionstackx
committed
fix tw-eth interface detection pattern in env.sh

authored and
functionstackx
committed
fix vllm-disagg config schema: use scenarios.fixed-seq-len

authored and
functionstackx
committed
fix vllm-disagg routing to multi_node benchmark subdir

authored and
functionstackx
committed
fix result collection to use FRAMEWORK as log directory prefix

authored and
functionstackx
committed
suppress tokenizer warnings and debug output in bench.sh

authored and
functionstackx
committed
fix vllm-disagg deadlock: stop router after rank 0 container exits

authored and
functionstackx
committed
reduce vllm-disagg concurrency sweep to single point for faster iteration

authored and
functionstackx
committed
preserve slurm logs on failure and print stderr inline

authored and
functionstackx
committed
enable set -x around docker privilege detection for CI debugging

authored and
functionstackx
committed
fix docker detection: test on compute node, not batch host

authored and
functionstackx
committed
fix docker detection: per-node probe since group membership varies

authored and
functionstackx
committed
add vllm-disagg changelog entries and update kimi conc-list

authored and
functionstackx
committed
switch vllm-disagg to 8k1k config to trigger multi-node eval

authored and
functionstackx
committed
add multi-node eval feature

ichbinblau
authored and
functionstackx
committed
remove start_etcd.sh

ichbinblau
authored and
functionstackx
committed
change decode to 1, easier for testing

ichbinblau
authored and
functionstackx
committed
add --served-model-name to vllm serve commands and wire up eval

authored and
functionstackx
committed
fix model name consistency between vllm serve and bench client

authored and
functionstackx
committed
add token patch to bench for vllm

ichbinblau
authored and
functionstackx
committed
add --tokenizer passthrough to run_benchmark_serving

authored and
functionstackx
committed
update vllm image for kimi2.5 and Minimax disagg.

ichbinblau
authored and
functionstackx
committed
Update setup_deps.sh

ichbinblau
authored and
functionstackx
committed
Update amd-master.yaml

ichbinblau
authored and
functionstackx
committed
update req rate for vllm.

ichbinblau
authored and
functionstackx
committed
make the sglang env consistent with upstream

ichbinblau
authored and
functionstackx
committed
node blacklist

ichbinblau
authored and
functionstackx
committed
fix: remove faulty minimax patch

simondanielsson
authored and
functionstackx
committed
fix: remove unneeded commented-out code from setup_deps.sh

simondanielsson
authored and
functionstackx
committed
fix: bump to latest nightly vllm image on minimax

simondanielsson
authored and
functionstackx
committed
fix: temporarily mount /coredumps

simondanielsson
authored and
functionstackx
committed
tmp: add bette r debugging capabilities

simondanielsson
authored and
functionstackx
committed
fix: disable custom all-reduce for minimax

simondanielsson
authored and
functionstackx
committed
fix: minimax segfault by avoiding M=8K fmoe kernel shape

simondanielsson
authored and
functionstackx
committed
revert: fix: temporarily mount /coredumps

simondanielsson
authored and
functionstackx
committed
feat: add VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT=1 as in single node example

simondanielsson
authored and
functionstackx
committed
fix: use FRAMEWORK arg in collect_latest_results.py to match vllm-disagg log dirs

ichbinblau
authored and
functionstackx
committed
remove unused vllm_disagg_utils directory

authored and
functionstackx
committed
revert: restore backend_request_func.py to match main

authored and
functionstackx
committed
revert: restore benchmark_serving.py to match main

authored and
functionstackx
committed
revert: fully restore benchmark_serving.py to match main

authored and
functionstackx
committed
revert: fully restore backend_request_func.py to match main

authored and
functionstackx
committed
add pr-link to vllm-disagg changelog entries

authored and
functionstackx
committed
fix: sync env.sh with upstream main

authored and
functionstackx
committed
fix: restore SGLANG_MORI_COMBINE_DTYPE in server launch commands

authored and
functionstackx
committed
refactor: move static vLLM env vars to env.sh, remove dead etcd code

authored and
functionstackx
committed
fix: pass IS_MULTINODE into Docker container

authored and
functionstackx
committed
fix: improve vllm-disagg changelog descriptions

authored and
functionstackx
committed
fix: restore DP+EP override blocks and trailing newline in server_sglang.sh

authored and
functionstackx
committed