Skip to content

Add steering input queue drained at the tool-call boundary - #3

Open
acoliver wants to merge 1 commit into
mainfrom
feature/steering-input
Open

acoliver wants to merge 1 commit into
mainfrom
feature/steering-input

Conversation

@acoliver

@acoliver acoliver commented Sep 5, 2026

Copy link
Copy Markdown
Owner

A message typed while an agent is working cannot currently reach the running turn. AgentStream exposes events and cancel() and nothing else, so a caller who wants to redirect the agent has to kill the turn and throw away the work it has already produced.

This adds the input channel, drained at the tool-call boundary: after a step's tool returns are appended to the history and before the loop issues the next model request. That is the same boundary Claude Code uses.

API

let queue = SteeringQueue::new();
let options = RunOptions::new().steering(queue.clone());

// from any task, while the agent is working:
queue.steer("focus on the error case".to_string());
  • SteeringQueue: cheaply cloneable, Default, backed by a tokio unbounded mpsc. steer(String) -> bool returns false only if the queue is closed, so input cannot be lost silently. pending_len() reports texts enqueued but not yet delivered.
  • RunOptions::steering(queue), mirroring message_history.
  • AgentStreamEvent::SteeringDelivered { step, text }, emitted once per delivered text, after that step's ToolExecuted events and before the next RequestStart, so a consumer can persist the user message in transcript order.

SteeringReceiver is pub(crate). A run claims the single receiver for its lifetime, which makes "one consumer" a property of the type rather than a rule in the docs.

Delivery rules

Nothing is drained before the first model request, and nothing is drained at a text-only end of turn. A run that never crosses a tool-call boundary delivers nothing and leaves the text queued. The receiver parks itself back on the originating queue when dropped, so leftovers survive the run that claimed them and a later run from the same queue delivers them at its first boundary. All four rules are in the rustdoc with a compiling doctest.

Both spawn loops drain. AgentStream::new and AgentStream::new_with_cancel each carry their own copy of the agent loop, and a drain added to only one would work in every test that uses new while silently doing nothing for callers that use new_with_cancel. The drain itself is one shared helper, deliver_queued_steering, called from both.

Breaking change and version

AgentStreamEvent is not #[non_exhaustive], so the new variant is source-breaking for downstream match sites that enumerate every variant without a wildcard. Following the CHANGELOG convention for this 0.x workspace, that is a minor bump: 0.3.0 to 0.4.0 across all crates, as 0.3.0 itself was released. The bump is a proposal, not a decision. Say the word and I will re-cut it however suits your release process.

No dependency was added. parking_lot was already declared for serdes-ai-agent.

Tests

Six unit tests, five integration tests, one doctest. The integration tests steer from inside the tool executor closure, so the text genuinely arrives mid-turn rather than being pre-loaded before the run:

  • delivery lands in the second model request, asserted against the mock's captured request history, with the first request confirmed not to contain the steered text, and event order asserted by position (ToolExecuted < SteeringDelivered < next RequestStart)
  • multiple steers deliver FIFO at one boundary, including that nothing drains before the first request
  • a text-only run delivers nothing and pending_len() still reports the text
  • a leftover from a text-only run is delivered by a follow-up run from the same queue
  • the new_with_cancel path delivers at the boundary

No sleeps; enqueueing from the tool closure makes the ordering deterministic.

Verification

The workflow's exact commands: cargo check --workspace --all-features, cargo fmt --all -- --check, cargo clippy --workspace --all-features -- -D warnings, cargo test --workspace --all-features. All pass, 1388 passed / 0 failed.

Note on the base branch

This targets feature/issue-65-open-responses rather than main because PersonalAgent pins all eight serdes-ai-* crates to a single rev, and serdes-ai-responses exists only on that branch. Once janfeddersen-wq#66 lands upstream this rebases onto main as a standalone change; it has no dependency on the Responses work beyond sharing a base.

Separately, and not addressed here: e675674 on the base branch committed tmp/ocr-pr66/review.json (76 KB), tmp/ocr-pr66/review.stderr and tmp/commitmsg.txt, which look like scratch files. The fork's .gitignore covers *.log but not tmp/. Worth removing before janfeddersen-wq#66 goes upstream.

@acoliver

acoliver commented Sep 5, 2026

Copy link
Copy Markdown
Owner Author

Note on CI: no checks will appear on this PR, and that is expected rather than a failure. .github/workflows/ci.yml triggers on pull_request: branches: [main], and this PR targets feature/issue-65-open-responses, so the workflow never fires. GitHub reports "no checks reported on the branch".

The workflow's four commands were run locally against this branch instead, in the same order the workflow runs them:

cargo check --workspace --all-features      # with RUSTFLAGS=-Dwarnings, as the workflow env sets
cargo fmt --all -- --check
cargo clippy --workspace --all-features -- -D warnings
cargo test --workspace --all-features

All four pass. The test run was 1388 passed, 0 failed, 128 ignored, across 39 test binaries, with the ignored set unchanged from the base commit.

Once janfeddersen-wq#66 lands upstream and this rebases onto main, the workflow will run for real and the result will be visible rather than asserted.

Downstream consumers drive agent turns through AgentStream, which until
now exposed only events and cancel(). A message typed while the agent is
working had no way to reach the running turn: the caller had to wait for
the current run to finish and start a new one.

This adds SteeringQueue, a cheaply cloneable handle over a tokio
unbounded mpsc. The caller keeps a clone wherever input arrives and
passes another to the run via RunOptions::steering, mirroring how
message_history is attached. The stream claims the queue's single
receiver for the run and drains it, FIFO and without blocking, at the
tool-call boundary: after the step's tool returns join the history and
before the next model request. That is the same boundary Claude Code
uses, and it is the only point where the model is guaranteed to see
fresh user intent without corrupting message alternation.

Each drained text becomes its own ModelRequest with a user prompt part
and emits AgentStreamEvent::SteeringDelivered { step, text } after the
step's ToolExecuted events and before the next RequestStart, so a
consumer can persist the user message in transcript order. Nothing is
drained before the first model request, and a run that never crosses a
tool boundary delivers nothing; leftovers stay queued and a later run
from the same queue delivers them, so the queue survives turns rather
than silently eating input.

Both AgentStream::new and AgentStream::new_with_cancel share the drain
through one helper because the two spawned loops are physically
distinct bodies and steering must behave identically in each.

AgentStreamEvent is not #[non_exhaustive], so the new variant is
source-breaking for exhaustive matches; per the workspace convention
that means a minor bump to 0.4.0 across crates, recorded in the
changelog.
@acoliver
acoliver force-pushed the feature/steering-input branch from c0e7327 to 9db0235 Compare September 6, 2026 01:49
@acoliver
acoliver changed the base branch from feature/issue-65-open-responses to main September 6, 2026 01:49
@acoliver acoliver closed this Sep 6, 2026
@acoliver acoliver reopened this Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant