[Feat]: first impliment for async GPU connector - #239

Closed

specture724 wants to merge 18 commits into

vllm-project:mainfrom

specture724:AsyncGPU

Commits on Aug 17, 2026

init: first impliment for async GPU connector
specture724
committed
896f30b
fix: flash_comm_v1_enabled for NPU only
specture724
committed
71892a3
perf: gather dispatch rows into the peer window, poll off the compute stream

specture724
and
claude
committed
69bf40b
perf: drop the per-work-item device synchronizes from the FFN path

specture724
and
claude
committed
84bcd6c
perf: ship each token once per destination and reduce on the FFN side

specture724
and
claude
committed
5b20d7c
refactor: fold the two receive spin loops into SymmWindow.wait

specture724
and
claude
committed
f2c9a08
feat: run async MoE ubatching on GPU

specture724
and
claude
committed
1f38e07
perf: weight and reduce the expert output in the payload dtype

specture724
and
claude
committed
741db0d
perf: wait for the expert reply on the stream, not on the host

specture724
and
claude
committed
a3421e4
perf: resolve the Ascend DBO yield once instead of per MoE layer

specture724
and
claude
committed
32aeace
perf: cut the per-layer host work on the dispatch path

specture724
and
claude
committed
cbb8a24
perf: read the dispatch plan back into pinned memory

specture724
and
claude
committed
3f8c6b0
perf: find the expert boundaries with searchsorted, not bincount

specture724
and
claude
committed
a094760
test: give the lifecycle connector fake an extra_info

specture724
and
claude
committed
ce2a6ac
perf: build dispatch headers on the device and write slots at capacity
specture724
committed
6d26165

Commits on Aug 26, 2026

style: reformat the window roundtrip smoke test

specture724
and
claude
committed
8a0e7b9
perf: size the shared field by the per-rank split, not by the batch

specture724
and
claude
committed
48ac7ec
fix: make the shutdown broadcast callable

specture724
and
claude
committed
c26c3da