MoE Prefill Streaming & DDTree Batched Verify#349

Merged

davide221 merged 9 commits into

Luce-Org:mainfrom

Jun 10, 2026

Commits on Jun 10, 2026

feat(dflash): MoE prefill streaming engine with mmap + batched GPU eval

howard0su
authored and
mrciffa
committed
perf(dflash): skip CPU when all routed experts are in VRAM (bitmask fast path)

howard0su
authored and
mrciffa
committed
perf(dflash): add cached hot-only batched graph for prefill sub-batches

howard0su
authored and
mrciffa
committed
perf(dflash): skip MMQ sub-batch workaround on sm_80+ (full-batch prefill)

howard0su
authored and
mrciffa
committed
perf(dflash): batched MoE verify for DDTree speculative decode

howard0su
authored and
mrciffa
committed
fix(dflash): address PR review comments (P1 + P2)

howard0su
authored and
mrciffa
committed
fix(gguf_mmap): add missing <cstdint> include
mrciffa
committed
fix(moe): gate MMQ full-batch mul_mat_id by batch size on reduced hot stacks
mrciffa
committed
fix(moe): integrate streaming storage with mainline LRU expert cache
mrciffa
committed