Skip to content

fix(broker): defer manual delivery ACKs #1243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
willwashburn merged 4 commits into from
Jul 14, 2026
Merged

fix(broker): defer manual delivery ACKs #1243

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
581a4d3
fix(broker): recover delivery cursor on resume (#1240)
willwashburn Jul 10, 2026
50ae9dc
fix(broker): defer manual delivery ACKs (#1241)
willwashburn Jul 10, 2026
f3f032a
style: auto-format with Prettier
github-actions[bot] Jul 10, 2026
6f312d9
merge: resolve conflicts with main for PR #1243
claude Jul 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .agentworkforce/trajectories/completed/2026-07/traj_cbx77hjzvdaz/summary.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Trajectory: Fix manual-flush Relaycast ACK durability

> **Status:** ✅ Completed
> **Task:** #1241
> **Confidence:** 93%
> **Started:** July 10, 2026 at 10:15 AM
> **Completed:** July 10, 2026 at 10:15 AM

---

## Summary

Fixed #1241 by separating received and ACKed delivery cursors, retaining Relaycast receipts through manual flush, rejecting full queues without eviction, and ACKing only successfully injected FIFO prefixes. Verified focused cursor, replay, overflow, success, and failure regressions; cargo fmt check passed; full broker library suite passed 672 tests with 4 ignored.

**Approach:** Standard approach

---

## Key Decisions

### Split delivery progress into received and ACKed cursors

- **Chose:** Split delivery progress into received and ACKed cursors
- **Reasoning:** Manual-flush must accept multiple contiguous Relaycast sequences without cumulatively acknowledging volatile queue entries; duplicates and gaps therefore report only the ACKed cursor, while successful enqueue advances received state.

### Reject the newest delivery when the manual queue is full

- **Chose:** Reject the newest delivery when the manual queue is full
- **Reasoning:** Evicting an already-held delivery could discard the only actionable copy after a later cumulative ACK. Atomic rejection leaves the FIFO unchanged, does not advance received or ACKed state, and preserves Relaycast ownership for replay.

---

## Chapters

### 1. Work

_Agent: default_

- Split delivery progress into received and ACKed cursors: Split delivery progress into received and ACKed cursors
- Reject the newest delivery when the manual queue is full: Reject the newest delivery when the manual queue is full
69 changes: 69 additions & 0 deletions .agentworkforce/trajectories/completed/2026-07/traj_cbx77hjzvdaz/trajectory.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
{
"id": "traj_cbx77hjzvdaz",
"version": 1,
"task": {
"title": "Fix manual-flush Relaycast ACK durability",
"source": {
"system": "plain",
"id": "#1241"
}
},
"status": "completed",
"startedAt": "2026-07-10T14:15:30.303Z",
"completedAt": "2026-07-10T14:15:39.949Z",
"agents": [
{
"name": "default",
"role": "lead",
"joinedAt": "2026-07-10T14:15:35.758Z"
}
],
"chapters": [
{
"id": "chap_xdlfc8rczkfx",
"title": "Work",
"agentName": "default",
"startedAt": "2026-07-10T14:15:35.758Z",
"endedAt": "2026-07-10T14:15:39.949Z",
"events": [
{
"ts": 1783692935759,
"type": "decision",
"content": "Split delivery progress into received and ACKed cursors: Split delivery progress into received and ACKed cursors",
"raw": {
"question": "Split delivery progress into received and ACKed cursors",
"chosen": "Split delivery progress into received and ACKed cursors",
"alternatives": [],
"reasoning": "Manual-flush must accept multiple contiguous Relaycast sequences without cumulatively acknowledging volatile queue entries; duplicates and gaps therefore report only the ACKed cursor, while successful enqueue advances received state."
},
"significance": "high"
},
{
"ts": 1783692935760,
"type": "decision",
"content": "Reject the newest delivery when the manual queue is full: Reject the newest delivery when the manual queue is full",
"raw": {
"question": "Reject the newest delivery when the manual queue is full",
"chosen": "Reject the newest delivery when the manual queue is full",
"alternatives": [],
"reasoning": "Evicting an already-held delivery could discard the only actionable copy after a later cumulative ACK. Atomic rejection leaves the FIFO unchanged, does not advance received or ACKed state, and preserves Relaycast ownership for replay."
},
"significance": "high"
}
]
}
],
"retrospective": {
"summary": "Fixed #1241 by separating received and ACKed delivery cursors, retaining Relaycast receipts through manual flush, rejecting full queues without eviction, and ACKing only successfully injected FIFO prefixes. Verified focused cursor, replay, overflow, success, and failure regressions; cargo fmt check passed; full broker library suite passed 672 tests with 4 ignored.",
"approach": "Standard approach",
"confidence": 0.93
},
"commits": [],
"filesChanged": [],
"projectId": "AgentWorkforce/relay",
"tags": [],
"_trace": {
"startRef": "3cfe0bf69bb24ad966b58ed288813bb3a7ef1a73",
"endRef": "3cfe0bf69bb24ad966b58ed288813bb3a7ef1a73"
}
}
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- `agent-relay-broker` resumes Relaycast mailbox delivery at the server's authoritative per-agent ACK cursor after a broker restart, preserving strict gap detection and legacy node compatibility.
- `agent-relay-broker` no longer acknowledges Relaycast `manual_flush` deliveries while they exist only in volatile memory; flushes ACK only an injected FIFO prefix, and full queues reject new deliveries without evicting held messages.
Comment on lines +104 to +105

@coderabbitai coderabbitai Bot Jul 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n '^## \[Unreleased' CHANGELOG.md
rg -n '^## \[9\.2\.2\]' CHANGELOG.md

Repository: AgentWorkforce/relay

Length of output: 211

🏁 Script executed:

#!/bin/bash
sed -n '1,130p' CHANGELOG.md | cat -n

Repository: AgentWorkforce/relay

Length of output: 17337

Move these bullets into ## [Unreleased - Patch]

CHANGELOG.md:104-105 already has an unreleased section at the top; leaving these agent-relay-broker fixes under ## [9.2.2] - 2026-07-08 makes that past release note claim changes that belong in the current patch train.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@CHANGELOG.md` around lines 104 - 105, Move both agent-relay-broker bullets
from the 9.2.2 release section into the existing ## [Unreleased - Patch] section
at the top of CHANGELOG.md, preserving their wording and order.

Source: Coding guidelines

- PTY snapshots (`view`/`drive`/`passthrough` attach, `GET /api/spawned/{name}/snapshot` ansi format) now capture and replay terminal modes — alt-screen, cursor visibility, application cursor keys, bracketed paste, mouse reporting, autowrap, and keypad — so attaching to a TUI no longer leaves the client terminal mis-configured (stray cursor, misbehaving arrows, broken paste). Each mode is re-emitted in both directions so an attach after a crashed session heals a terminal left in the wrong state.
- `agent-relay view`/`drive`/`passthrough`: detaching now emits a conservative terminal reset (leave alt-screen, show cursor, disable mouse reporting + bracketed paste + application cursor keys, reset scroll region) on TTY stdout, so a driven session's replayed snapshot and live stream can't leave your shell in a broken terminal state.
- `agent-relay drive`/`passthrough`: a `Ctrl+C` during attach setup no longer strands the worker's inbound delivery mode — an interrupt in that window can't leave the worker stuck in `manual_flush` (drive) or cancel an explicit `agent message hold` (passthrough).
Expand Down
10 changes: 6 additions & 4 deletions crates/broker/src/listen_api.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2149,10 +2149,10 @@ async fn listen_api_get_pending(

/// `POST /api/spawned/{name}/flush` → `{ "flushed": N }`.
///
/// Drains the queue and injects each message into the worker in order
/// using the existing fire-and-forget inject path. The inbound delivery mode is
/// *not* changed — a caller still in `manual_flush` delivery mode will continue
/// to queue newly-arriving messages.
/// Injects queued messages into the worker in FIFO order and stops at the first
/// failure, retaining that message and its suffix for a later attempt. The
/// inbound delivery mode is *not* changed — a caller still in `manual_flush`
/// delivery mode will continue to queue newly-arriving messages.
async fn listen_api_flush_pending(
axum::extract::State(state): axum::extract::State<ListenApiState>,
axum::extract::Path(name): axum::extract::Path<String>,
Expand Down Expand Up @@ -5064,6 +5064,7 @@ mod auth_tests {
mode: MessageInjectionMode::Steer,
queued_at_ms: 100,
event_id: Some(EventId::new("evt_1")),
relaycast_receipt: None,
},
PendingRelayMessage {
from: "Bob".to_string(),
Expand All @@ -5076,6 +5077,7 @@ mod auth_tests {
mode: MessageInjectionMode::Wait,
queued_at_ms: 200,
event_id: None,
relaycast_receipt: None,
},
]));
}
Expand Down
Loading
Loading