Conversation
vLLM (>= the DeepSeek-V4.1 day-0 image) hashes prefixes every prefix_match_unit (hash_block_size) tokens, which can be finer than the cache block size, and publishes the prompt tail that ends inside a cache block as a BlockStored whose block_size is the sub-block length (32/64/96 for a 128-token block). The standalone indexer treated any block_size mismatch on a main-attention event as a fatal --block-size misconfig and exited, so every V4.1 indexer crash-looped. Treat a proper divisor of the configured block size as a partial-prefix entry: drop it (rate-limited warn). Non-divisor mismatches stay fatal.
…ered pod Pod discovery registered each engine pod once, as dp_rank 0 on tcp://<ip>:<zmq_port>. vLLM offsets the KV-event ZMQ port and the /kv_recover port by data_parallel_rank, so on a --data-parallel-size N engine ranks 1..N-1 publish to ports nobody subscribes to and everything they cache is invisible to /query while the engine still hits it. Measured on deepseek-ai/DeepSeek-V4.1-Flash (DP=2): every prefill the scheduler placed on rank 1 (about half) showed 0 in the standalone indexer and in rank 0's /kv_recover dump, and all its blocks in rank 1's dump; the KV ladder read actual 0.78 > h24 0.65 > perfect 0.63 > reality 0.59. Add --watch-dp-size (default 1, so single-rank engines are unchanged). The watcher registers one listener per rank under the pod's instance: dp_rank r, tcp://<ip>:<zmq_port + r>, http://<ip>:<recover_port + r>. register() rejects a rank that is already present, so a pod whose registration fails part-way is deregistered before the retry instead of being left half-subscribed forever. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
standalone-indexer: subscribe to every data-parallel rank of a discovered pod
Shang-Pin
had a problem deploying
to
external_collaborator
September 17, 2026 21:15 — with 
GitHub Actions
Failure
GitHub Actions
Failure
github-actions
Bot
added
the
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
vLLM (from the DeepSeek-V4.1-Flash day-0 image,
vllm/vllm-openai:deepseekv41-flash-0909) hashes prefixes everyprefix_match_unit(hash_block_size, 32 for V4.1) tokens, which is finer than the cache block size (128). It publishes the prompt tail that ends inside a cache block as aBlockStoredon the main MLA group whoseblock_sizeis the sub-block length (32/64/96).The standalone indexer treats any main-attention block-size mismatch as a fatal
--block-sizemisconfiguration and callsstd::process::exit(1), so all threekv-indexer:*flavors fordeepseek-ai/DeepSeek-V4.1-Flashcrash-looped (engine block size 32 != configured --block-size 128). No indexer--block-sizeworks: 128 dies on the partial events, 32 dies on the whole-block events.Observed event mix on a V4.1 pod (150 s tap): groups 0-3
sliding_window_mla@32 (already filtered as non-main), group 4mla_attention@128 (whole blocks) plus @32 partial entries.Fix
convert_event: whenblock_size < kv_block_sizetreat the event as a partial-prefix entry and drop it (rate-limited warn) instead of erroring. A larger event block size stays fatal (that can only be a misconfig). Whole-block routing information is unaffected; only sub-block tail entries are ignored.Unit test added;
cargo test --lib zmq_wire::tests= 19 passed.Same commit cherry-picked onto
24h-indexerasv41-partial-skip-h24for the h24 image.Refs DEE-640.