Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions docs/designs/send-message-routing-rework.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ type AgentTarget = {
| {
agentId: string
needsReply?: boolean
deadlineMs?: number
}
channel?: string
message: string
Expand Down Expand Up @@ -197,6 +198,58 @@ the trusted caller session instead of treating a platform channel supplied by th
model as authorization. The child retains origin lineage, hop count, optional
correlation, `needsReply`, and `viewSessionStatus` support.

### 3.1a `needsReply` deadlines — making silence an event

`needsReply` has no timeout, so a child that simply never answers produces no
event anywhere. The parent ended its turn expecting a report; nothing wakes it
again, and `viewSessionStatus` is poll-only, which requires already being awake.
Measured in the webchat Werewolf arena: after the referee announced a vote and
ended its turn, votes only ever arrived after a **human** posted "the vote has
gone quiet" — 2–4 times in every completed game. A referee could not run its own
non-voter re-prompt lever, because nothing told it that anything was missing.

The #800 inferred reply covers the adjacent case — a child whose turn _ends_
without a report has its final output delivered to the parent. It cannot cover a
child that never starts, never finishes, or whose wake is gated: there is no turn
end to hang an inference on.

`toAgent.deadlineMs` (only with `needsReply: true`, 1s–24h) arms a one-shot,
child-anchored deadline. On expiry the daemon wakes the parent session with a
notice naming the target, the delivery ID, and the child's last known state. The
notice explicitly states it is not the child's answer — the daemon never
fabricates a reply. The parent decides: re-prompt, escalate, or proceed.

The mechanism reuses the retained orchestration-deadline machinery (§3.4/§6.8) —
a durable epoch, a live one-shot timer, a CAS claim, duty gating, and re-arm from
the store on startup and on every duty change — but keeps its own record, keyed
by child session:

- the durable row is written at **call** time, so it exists even when the child
has no session row and never gets one (the case the deadline exists for). It
also carries the child's coordinates for the same reason;
- a report arriving first disarms it (`markChildParentReply`), so a normal
delegation never pays for it;
- exactly-once between an arriving report and the firing timer is the CAS delete:
whichever runs first is the only one that acts, and on a shared store only one
pool member wins;
- the wake carries the parent as a trusted internal origin, because the ordinary
authorization reads the child's session row, which may not exist.

**Only where this member can actually fire it.** Every disarm path runs on the daemon that OWNS the
child, so a deadline armed on the caller for a cross-daemon target would never be
cancelled by an accepted remote report and would later fire a false "no report
arrived". Arming it on the child's daemon instead requires carrying the deadline and
durable parent routing through the relay — a wire change. Until then a `deadlineMs` on
a remote target is refused loudly, not silently ignored: the daemon logs it and the
tool result carries `deadlineIgnored`, so the caller falls back to
`viewSessionStatus` rather than waiting for a wake that will never come.

The deadline is **parent-owned**: the wake dispatches into the CALLER's session, so the
caller's duty holder is the member that must arm and fire it, and the durable row carries
`parentAgentId` for exactly that gate. Where the CHILD runs is irrelevant to ownership. If
this member does not hold the caller's duty, the same loud refusal applies
(`caller_duty_elsewhere`) rather than arming a timer nothing will fire.

### 3.2 Channel-root form

`{"toAgent":"<agent-id>","channel":"<channel-id>","message":"..."}`:
Expand Down
50 changes: 49 additions & 1 deletion evals/games/night-collection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,13 @@ export interface NightCollectionRefereeConfig {
wolfB: WebchatSeat
seer: WebchatSeat
doctor: WebchatSeat
/** #800 deadline attached to every night call. Absent ⇒ the pre-deadline behavior. */
deadlineMs?: number
}

/** The marker text of the daemon's deadline wake, as the referee sees it. */
export const DEADLINE_NOTICE = '[needsReply deadline]'

const instruction = (task: string, marker: string): string =>
`${task} Answer with a single line that starts exactly with \`${marker}\` — nothing before it. ` +
`Do not contact anyone else and do not post anywhere.`
Expand All @@ -62,6 +67,9 @@ export class NightCollectionReferee implements ScriptedBrain {
readonly issued: IssuedCall[] = []
/** Marker → number of onPrompt() calls whose text contained the reply. */
readonly markerSightings = new Map<NightMarker, number>()
/** Deadline wakes seen, and the re-prompts they let the referee send unaided (#800). */
deadlineNotices = 0
readonly rePrompted = new Set<NightMarker>()
private nightIssued = false
private relayIssued = false
private closed = false
Expand Down Expand Up @@ -109,13 +117,46 @@ export class NightCollectionReferee implements ScriptedBrain {
)
)
}
// #800: the deadline wake is the ONLY thing that reaches a referee whose child went
// silent, and it is what makes an unaided re-prompt possible.
if (text.includes(DEADLINE_NOTICE)) {
this.deadlineNotices += 1
for (const [alias, purpose] of this.seatPurposes()) {
if (!text.includes(alias) || this.rePrompted.has(purpose)) continue
this.rePrompted.add(purpose)
calls.push(
this.needsReplyCall(
this.seatFor(purpose),
purpose,
instruction(`You did not answer. Send your night action now.`, MARKERS[purpose])
)
)
}
}
if (!this.closed && text.includes(MARKERS.verdict)) {
this.closed = true
reply = 'The night is resolved.'
}
return { calls, reply }
}

private seatFor(purpose: NightMarker): WebchatSeat {
return purpose === 'proposal'
? this.cfg.wolfA
: purpose === 'verdict'
? this.cfg.wolfB
: purpose === 'seer'
? this.cfg.seer
: this.cfg.doctor
}

private seatPurposes(): [string, NightMarker][] {
return (['proposal', 'verdict', 'seer', 'doctor'] as NightMarker[]).map((purpose) => [
this.seatFor(purpose).agentId,
purpose
])
}

onCallResult(outcome: BrainCallOutcome): void {
const toAgentId = (outcome.args.toAgent as { agentId?: string } | undefined)?.agentId
const row = this.issued.find((candidate) => candidate.to === toAgentId && !this.settled.has(candidate))
Expand All @@ -133,7 +174,14 @@ export class NightCollectionReferee implements ScriptedBrain {
this.issued.push({ to: seat.agentId, purpose, needsReply: true, delivered: false })
return {
tool: 'sendMessage',
args: { toAgent: { agentId: seat.agentId, needsReply: true }, message }
args: {
toAgent: {
agentId: seat.agentId,
needsReply: true,
...(this.cfg.deadlineMs !== undefined ? { deadlineMs: this.cfg.deadlineMs } : {})
},
message
}
}
}
}
Expand Down
70 changes: 67 additions & 3 deletions evals/test/webchat-night-collection.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,15 @@ import { callDaemonTool } from '../games/mcp-client.js'

const ALIASES = NIGHT_ALIASES

async function waitUntil(predicate: () => boolean, timeoutMs: number): Promise<void> {
const deadline = Date.now() + timeoutMs
while (Date.now() < deadline) {
if (predicate()) return
await new Promise((resolve) => setTimeout(resolve, 100))
}
throw new Error(`condition not reached within ${timeoutMs}ms`)
}

interface NightRun {
arena: WebchatArena
seats: WebchatSeat[]
Expand All @@ -49,14 +58,27 @@ interface NightRun {
* force a child reply to land while the referee's turn is still in flight
* (the coalesce cell).
*/
async function startNightRun(options: { refereeGate?: (promptText: string) => Promise<void> } = {}): Promise<NightRun> {
async function startNightRun(
options: {
refereeGate?: (promptText: string) => Promise<void>
/** #800 deadline the referee attaches to every night call. */
deadlineMs?: number
/** Aliases whose delegation turn NEVER ends — the shape no turn-final inference reaches. */
silent?: string[]
} = {}
): Promise<NightRun> {
const seats = mintSeats([...ALIASES])
const seat = (alias: (typeof ALIASES)[number]) => seats.find((candidate) => candidate.alias === alias)!
const referee = new NightCollectionReferee({
wolfA: seat('wolf-a'),
wolfB: seat('wolf-b'),
seer: seat('seer'),
doctor: seat('doctor')
doctor: seat('doctor'),
...(options.deadlineMs !== undefined ? { deadlineMs: options.deadlineMs } : {})
})
let releaseSilent: () => void = () => undefined
const silentGate = new Promise<void>((resolve) => {
releaseSilent = resolve
})
const log: PromptLogEntry[] = []
const handlers = new Map<string, ScriptedSessionHandler>()
Expand Down Expand Up @@ -90,6 +112,13 @@ async function startNightRun(options: { refereeGate?: (promptText: string) => Pr
return undefined
})
handlers.set(seat('villager').agentId, ({ text }) => (text.includes('NIGHT 1 begins') ? 'Waiting.' : undefined))
for (const alias of options.silent ?? []) {
handlers.set(seat(alias as (typeof ALIASES)[number]).agentId, async ({ text }) => {
if (text.includes('NIGHT 1 begins')) return 'Waiting.'
await silentGate
return undefined
})
}

const { root } = prepareScriptedWebchatRoot(seats)
const arena = new WebchatArena({
Expand Down Expand Up @@ -119,7 +148,10 @@ async function startNightRun(options: { refereeGate?: (promptText: string) => Pr
{ alias: 'wolf-b', marker: 'verdict' }
]
}),
stop: () => arena.stop()
stop: async () => {
releaseSilent()
await arena.stop()
}
}
}

Expand Down Expand Up @@ -224,4 +256,36 @@ describe('webchat night collection (scripted)', () => {
await run.stop()
}
}, 120_000)

it('#800 deadline: a child that never reports wakes the referee anyway, and it re-prompts unaided', async () => {
// The seer's delegation turn never ends, so nothing turn-final can infer a reply for it.
// Before the deadline this referee had no event to act on at all — the live game needed a
// human to say "the vote has gone quiet".
const run = await startNightRun({ deadlineMs: 2_000, silent: ['seer'] })
try {
await run.arena.postHost(NIGHT_START_TEXT)
// The silent child's turn never ends, so the arena never goes idle — poll for the
// recovery instead of waiting for a quiet that cannot come.
await waitUntil(() => run.referee.rePrompted.has('seer'), 60_000)

const refereeInput = run.refereePrompts().join('\n')
expect(refereeInput).toContain('[needsReply deadline]')
expect(refereeInput).toContain('No report arrived')
// The notice is not an answer: the seer's marker never appears through it.
expect(refereeInput).toContain('this notice is NOT its answer')
expect(run.referee.deadlineNotices).toBeGreaterThanOrEqual(1)

// …and the referee acted on it by itself — the lever a quiet child previously blocked.
expect(run.referee.rePrompted.has('seer')).toBe(true)
expect(run.referee.issued.filter((call) => call.purpose === 'seer')).toHaveLength(2)

// The children that DID report are unaffected.
const score = run.score()
for (const child of ['wolf-a', 'doctor']) {
expect(score.replies.find((reply) => reply.child === child)!.mode).not.toBe('lost')
}
} finally {
await run.stop()
}
}, 120_000)
})
Loading