Skip to content

[WIP] streaming graph walk transition#97

Open
NSagan271 wants to merge 16 commits into
mainfrom
nsagan-streaming-graph-walk-transition
Open

[WIP] streaming graph walk transition#97
NSagan271 wants to merge 16 commits into
mainfrom
nsagan-streaming-graph-walk-transition

Conversation

@NSagan271

@NSagan271 NSagan271 commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator

Lets a producer partition drive a consumer's graph-walk transitions via the data stream, instead of the conductor centrally advancing them. This decouples a consumer's walk from the conductor's (lagging) view and enables PD disaggregation of the qwen3-omni Talker.

Mechanism

  • PartitionDefinition.transition_source (STATE_MACHINE vs PRODUCER_TRIGGERED) declares who owns a partition's walk.
  • A Connection.consumer_walk_transition callback maps streamed items to WalkTransition markers; the consumer's StreamBuffer cuts a chunk boundary at each transition so a walk never straddles two chunks.
  • Per-edge produced_edge_idx/consumed_edge_idx counters round-trip through the conductor; index-based dedup (first-arrival-wins) + monotonic set_index keep the stream continuous across the prefill→decode worker seam (PD).
  • qwen3-omni: Thinker streams thinker_states carrying the markers; prefill states keep the Talker in talker_prefill (KV extend), the first thinker_decode state flips it to talker_last_prefill, then the second decode state causes a transition to talker_decode. New configs/qwen3omni_talker_pd_disag.yaml splits the Talker across prefill/decode workers.
  • Graph-walk transitions gated on a fully reset ("clean") worker graph.

Touches conductor/, streaming/, worker/, graph/, qwen3-omni model.

Testing

Testing in progress; tested on default qwen3 omni config, 2-GPU, thinker TP2, and a new configs/qwen3omni_talker_pd_disag.yaml splits both the Thinker and Talker across prefill/decode workers.

@NSagan271 NSagan271 force-pushed the nsagan-streaming-graph-walk-transition branch 2 times, most recently from 88ab350 to 9830d30 Compare June 7, 2026 21:45
@NSagan271 NSagan271 force-pushed the nsagan-streaming-graph-walk-transition branch from 0c5e9c0 to 1475c28 Compare June 12, 2026 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant