SIGSEGV in InputBatch::add_request serving a text completion on a synthetic DeepSeek-V4 fixture, ~50% of runs

[localai-org-maint-bot]

Row: -

A TEXT completion served through VllmServerMain against a synthetic DeepSeek-V4 GGUF fixture crashes on roughly half of runs, at one token as readily as at 260.

Two signatures were observed, both on the same harness:

• SIGSEGV in InputBatch::add_request
• otherwise SIGSEGV in gather_block_table

Observed 3 crashes in 6 runs while building the W5 serving gate for #2411. The IMAGE request path on the same harness reached ModelRegistry::Forward 8 times out of 8, so whatever this is, it is not the multimodal path.

It is reported rather than gated. The wave that found it removed the text case from its suite instead of asserting a behaviour it could not make deterministic — tests/vllm/entrypoints/openai/test_serve_deepseek_v4_mm.cpp therefore runs 2 cases with 1 skipped, and passes 8 of 8 for me at that shape. Asserting an unstable case would have made the suite a coin flip; leaving it unrecorded would have hidden a crash.

Two fixture conditions had to be pinned to get the load to happen at all, and they may be related or may be separate defects:

1. A GGUF without deepseek4.context_length resolves max_model_len = 0 and then SIGSEGVs, rather than refusing by name.
2. Prefix caching must be disabled: BlockPool::cache_full_blocks refuses this KV topology's block/hash-block pair.

What is NOT established: whether this reproduces on a real DeepSeek-V4 checkpoint, or only on the synthetic fixture; and whether it predates #2411 entirely. Nothing in W5 touches InputBatch or the block table, which is why this is filed separately from that row rather than as a regression against it. Establishing which of those two it is, is the first step, not the fix.

[localai-org-maint-bot]

Half of this is explained and fixed: the gather_block_table signature is an out-of-bounds read

full_attn_group_id_ is a -1 sentinel meaning "this model published no kFullAttention and no kMlaAttention group". DeepSeek-V4 publishes neither, so that sentinel is what every one of its served steps carries. The GDN gather beside it guards on its own sentinel; the full-attention gather did not, and MultiGroupBlockTable::operator[] casts its index to size_t — so block_tables[-1] read a BlockTable that does not exist.

The max_num_blocks_per_req it found there decided whether the request ran at all:

• a garbage 0 gathered an empty table and the request went on to the forward;
• a garbage negative made num_reqs * cols a ~1.8e19-element size_t, and the engine's busy loop died with std::length_error before any forward ran.

Which one came up moved with the binary's layout, not with the request. A probe reported both readings from one source tree: group_id=-1 cols=0 in the case order that passed, group_id=-1 cols=-1119040307 in the order that failed. Merging one branch into another flipped it, and so did running a single inert earlier case of the same suite ahead of it.

That is worth stating plainly because it means a reachability measurement taken against this path was resting on undefined behaviour returning a convenient zero — the request "reached the forward" because a garbage stride happened to be 0.

Fixed on the DeepSeek-V4 Flash Vision row (#2411): an out-of-range group now gathers an empty table, which is what the rest of the step is already written against — MakeCommonAttentionMetadata is handed the same sentinel one line later and its group < slot_mapping.size() is false for -1, so it leaves that group's slot mapping empty. Byte-neutral for every model that publishes a full-attention group, which is every model shipping today. 732 of 732 non-CUDA tests pass.

What is still open, and why this issue stays open

The InputBatch::add_request SIGSEGV is neither explained nor repaired. It is a different signature and it is not established that it shares a cause with the above.

Also unresolved, and possibly the real question underneath: whether DeepSeek-V4's one published KV group should be classified as the target attention group at all, rather than leaving the model permanently on the -1 sentinel. That is owed by row KV-DSV4-MULTICACHE W3 (#2068), not here.

Still not established from the original report: whether either signature reproduces on a real DeepSeek-V4 checkpoint or only on the synthetic fixture.