[5/5][WIP] feat: request-aligned DBO ubatch splitting (flag off by default)

[specture724]

Fifth and last of the async GPU connector stack. Stacked on #335 — review that
one first; this PR's diff is only the DBO change.

#	PR	lines
1	#332 symmetric window	1336
2	#333 async connector	3086
3	#334 DeepSeek-V4	1203
4	#335 FFN padded graphs	1190
5	this	529

What this does

Upstream vLLM splits a DBO batch at an even token count, cutting whichever
request straddles that point into both ubatches. AFD's overlap is between
whole requests — one request's dispatch is in flight while the other's
attention runs — so the split has to land on a request boundary, and a batch
with no interior boundary (a single prefill) must run whole.

The splitter also prefers 16-token-aligned boundaries: a ubatch's per-token
tensors are views starting at the split point, and DeepSeek-V4's CuTeDSL
compressor kernel rejects anything below 64-byte alignment. It is a
preference, not a rule — uniform decode has boundaries at 1, 2, 3, … and
refusing there would disable DBO for decode entirely.

Two bugs that kept DBO from working at all on this connector:

• the batch-execution override only consulted the rank-local decision when
  data_parallel_size == 1, but the cross-DP agreement is also skipped for
  connectors with no control plane, where the parent's False is a
  placeholder rather than a decision — so --enable-dbo was accepted and then
  silently ignored for the whole run;
• the last split point came off cu_num_tokens as a numpy.int32, which
  Triton refuses to specialise, killing DeepSeek-V4 in
  _build_c128a_topk_metadata_kernel on the last ubatch.

The dispatch stage is now the DBO ubatch id read from the thread, the way vLLM
tracks it; forward_context.ubatch_idx does not exist, so both halves looked
like stage 0 and claimed the same window slot.

Performance: DBO is a regression on every configuration measured

ENABLE_DBO defaults to 0 and should stay there. This PR makes the flag
work correctly; it does not make it profitable. Four-arm ablation,
DeepSeek-V2-Lite 1A1F decode, 2x L20X, 64 req x 256 output tokens, conc 16,
3 repeats in one server lifetime:

arm	mean	stdev	vs eager
eager	36.642 s	0.047	—
eager + DBO	79.573 s	1.581	−117%
graphs	28.345 s	0.049	+22.6%
graphs + DBO	53.826 s	0.983	−46.9%

On DeepSeek-V4 prefill it is −7.1% at 2048-token steps, −3.7% at 4096, and
+0.5% at 8192 with 4096-token prompts — i.e. break-even at best.

Note also that the +22.6% graph win above is V2-Lite-specific. The same
comparison on DeepSeek-V4 decode measured 137.5 s eager against 134.6 s with
graphs — about 2%, and the eager arm's spread (sd 6.1 s) covers it, so it is
not a significant difference. V4 is FFN-compute-bound, so neither lever has
much host time to reclaim there.

Why, and when it could pay

The cost is round-trip latency, not host CPU. Normalising to per-layer
dispatch round trips (26 MoE layers, ~1024 decode steps):

arm	step	round trips/layer	per round trip
eager	35.8 ms	1	1.376 ms
eager + DBO	77.7 ms	2	1.494 ms
graphs	27.7 ms	1	1.065 ms
graphs + DBO	52.6 ms	2	1.011 ms

A round trip costs the same whether DBO is on or off; DBO simply performs
twice as many per step while halving the work inside each. Under graphs the
per-round-trip cost is marginally lower with DBO than without — Python is
out of the loop by then — and the 1.90x remains. The overlap that is supposed
to hide one ubatch's round trip behind the other's compute hides essentially
nothing.

That gives a falsifiable criterion: per-ubatch compute must be at least the
~1 ms round-trip latency. On V2-Lite decode a ubatch is 8 sequences x 1
token, orders of magnitude short. Either the round trip drops to ~100 us, or
DBO needs a regime where a ubatch carries >=1 ms of compute.

Merging it keeps the correctness fixes and the splitter available for that
regime, with the flag off by default.

Known gaps

DeepSeek-V4 decode hangs with --enable-dbo. Three attempts to measure a
DSV4 decode ablation (2A2F, 4x L20X, batch 64, 8192-token steps) never produced
a DBO number. The current failure, with everything in this PR applied:

EngineCore_DP1 ... TimeoutError: RPC call to sample_tokens timed out.

The server starts and serves; it hangs once decode begins. The AFD ubatch
wrapper logs nothing in that run, so the hang precedes any ubatch split rather
than happening inside one. Not root-caused.

Two bugs were found and fixed along the way, both real but neither the cause:

• afd_plugin/__init__.py imported two patch modules that no longer exist, and
  all the imports shared one try, so the ModuleNotFoundError silently
  skipped ubatch_positions and ubatch_split — this PR's own patches never
  loaded. Each import is now isolated and failures warn instead of debug-log.
• RemoteDeepseekV4FFN read forward_context.ubatch_idx, which does not
  exist, so both DBO halves resolved to stage 0 and claimed the same window
  slot. V4 now uses the per-thread current_dbo_ubatch_id() like V2.

So ENABLE_DBO=1 is not usable on DeepSeek-V4 today. It defaults to 0 and no
default path is affected; the splitter and the two correctness fixes above are
what this PR is worth merging for.

Under DBO a prefill step never replays a captured graph: the per-ubatch
metadata comes from split_attn_metadata, which clones per step, so the
identity precondition in _refresh_metadata_pair refuses — instrumented at
replays=0. Forcing the shapes to match makes it replay and then segfault
inside cuGraphLaunch on the first launch. Decode is unaffected (98% replay
share, no crash), so this is specific to the cooperative prefill graph.
Investigation branch: afd/async-gpu-replay-safe.

Test plan

• pytest tests/unit — no failures beyond the 13 already present on [4/5] feat: capture the FFN experts at bucketed padded shapes #335
• pre-commit run --from-ref afd/async-gpu-cudagraph --to-ref HEAD — clean
• new unit coverage for the splitter (request alignment, 16-token preference,
  single-request refusal, decode fallback) and the DP-coordination gate
• ablation above run on real hardware

🤖 Generated with Claude Code

[jiangkuaixue123]

With DBO enabled, the decode execution time appears to nearly double. Could you share a profiling trace to check whether the two ubatches actually overlap? For prefill, could you also check whether it is host-bound, with a significant amount of time spent launching kernels, especially since graph replay is currently not working for DBO prefill? A CPU/GPU timeline would help distinguish a lack of overlap from host-side launch overhead.