Pipeline the first next with the dispatch task frame to collapse two round-trips into one

[conradbzura]

Description

WorkerConnection.dispatch performs two sequential client<->worker round-trips per dispatch: (1) write the Task frame and read the Ack (handshake), then (2) write the first next and read the first result. The client stalls a full round-trip waiting for the ack before sending the next, even though every dispatch pulls at least once — a coroutine pulls exactly once, and an async generator's first operation is always a pull.

Pipeline the first pull with the task: after writing the Task frame during the handshake, write the first next frame before reading the ack. The worker sends its ack, runs the routine to its first result, and returns that result without a second client round-trip. The DispatchStream for the call is told its first pull is already primed, so the first __anext__ reads the already-in-flight result without issuing another next. Only the first pull is pipelined; asend/athrow/subsequent next are unchanged, as is the chain-manifest snapshot carried on the first next. A prototype lives behind an EAGER_FIRST_NEXT flag in wool.runtime.worker.connection.

Motivation

On loopback, the prototype measured dispatch p50 latency -9.1% (858us -> 780us). This understates the real win: on loopback a "round-trip" is asyncio/grpc machinery, not network latency (a warm unary RTT measured ~215-293us), so the second round-trip is nearly free. On a real network with a non-trivial RTT, collapsing 2 RTTs -> 1 saves a full network round-trip, which can dwarf the local ~78us. It is one of two independent, additive dispatch-latency optimizations; the other (memoizing the proxy/callable pickle) is filed as #273.

Failure paths (verified, eager off vs on — byte-identical caller-observable outcomes):

• Parse-phase nack (e.g. non-async callable): the worker's DispatchSession.__aenter__ reads only the first request (the Task) and, on Rejected, ships a Nack and returns without reading a second request — the pipelined next is left unread and dropped silently (no worker-side warning/error). The caller reads the Nack and raises the original exception as today.
• Transport failure (dead connection / worker abort): the read surfaces the gRPC status (UNAVAILABLE -> TransientRpcError); writing the eager next to a dead/aborting stream does not mis-handle — a write that raises grpc.RpcError propagates into dispatch's existing except grpc.RpcError arm, otherwise the read surfaces the status. Same classification eager on/off.
• Post-ack routine exception: the primed pull reads the exception frame and raises it, identical to today.

Semantic to document: eager makes the worker begin executing the routine's first step as soon as the Task arrives (the next rides with it), before the caller's explicit first pull. This is protocol-safe (the first pull is mandatory) and invisible on all failure paths (a nack precedes execution; a transport failure never reaches the worker). The only observers are (a) code driving WorkerConnection.dispatch() directly that opens a stream and never iterates it, or (b) a dispatch cancelled in the narrow window between the handshake and the first pull — in both, eager will have run one worker-side step that non-eager left pending. Minor wasted work, not a correctness difference.

Stdlib parity is preserved and must be regression-locked: invoking a @wool.routine (some_routine() / some_gen()) without awaiting/iterating dispatches nothing — verified empirically with eager forced on (the worker stays idle across a 0.3s window; idle resets only on await / first iteration). This holds because eager only changes bytes emitted inside _handshake, which is inside the awaited wrapper body — there is no path from constructing the coroutine object to touching the network.

Expected Outcome

• Coroutine and async-generator dispatch produce identical caller-observable results with eager on vs off across: happy path, post-ack routine exception, parse-phase nack, and transport failure.
• On the nack path, the orphaned next is dropped cleanly on the worker — no warning, error, or resource leak.
• Only the first pull is pipelined; asend, athrow, and subsequent next frames are unchanged, and the chain-manifest snapshot on the first next is unchanged.
• A tests/stdlib_parity pin asserts "invoke-without-await/iterate dispatches nothing; await/iterate triggers execution," passing with eager enabled.
• The eager-execution-of-first-step semantic is documented.