Store MoE routing indices per-block in KVBlockAllocator for prefix caching sharing#2

Open

lmcafee-nvidia wants to merge 6 commits into

sidsingh-nvidia:siddharth/support-nemo-rl-router-replayfrom

lmcafee-nvidia:prefix-caching-router-record

Commits on Mar 19, 2026

Inference | Per-block MoE routing storage for prefix caching compatibility.

lmcafee-nvidia
and
claude
committed
Assert on routing/token count mismatch instead of silent return

lmcafee-nvidia
and
claude
committed
Convert per-block routing storage from Tensor to ndarray

lmcafee-nvidia
and
claude
committed

Commits on Apr 6, 2026

Commits on Apr 7, 2026

fix _router_record_bookkeeping to return dict keyed by request_id

authored and
root
committed