Should not have giant hacks to reuse sandbox for validation loop

[cgwalters]

#6753 (comment)

This code is really, really bad.

I know dropping the validation loop would be a big change, but the only other non-hacky way I can think of to fix this is to not terminate the agent process - the agent has a MCP tool it calls that says "I think I'm done" which runs the validation loop.

[waynesun09]

Sorry, my reply landed on the PR thread an hour after you moved here: #6753 (comment). Short version of it first, then your proposal.

Short version: agreed the sweep is a workaround. Your "don't terminate the agent" idea is the right shape, and #6959 already built most of it. Applied to validation it removes the sweep from the loop entirely.

What the PR reply says

• The gap is in OpenShell. Checked against the v0.0.116 source: sandbox exec gives back stdout, stderr and an exit code, nothing can signal the command or its children afterwards, and the SSH signal handler only reaches the sandbox main process group. Filed as feat(sandbox): let the caller terminate a sandbox exec command and everything it spawned NVIDIA/OpenShell#3159. When that lands, the sweep goes away at one seam.
• The deterministic check is already plain CI. The validation script runs on the host. What validation_loop adds is the feedback path: the retry continues from the same working tree with the failure attached.

Your proposal, mapped onto what exists

#6959 keeps one sandbox and one session per run and talks to the live session at turn boundaries. The same mechanics cover validation:

Runtime	What happens at turn end	Kill?
Claude Code, pi	Process stays alive on stdin. Runner downloads the output, runs the validation script on the host, then sends the failure as the next user message, or tells the session to finish on pass.	No
Codex	No live channel, so the process exits on its own and the runner runs codex exec resume <thread_id> on the same thread with the failure.	No

• The harness contract does not change. validation_loop.script, schema, max_iterations and feedback_mode stay as they are; only what a "retry" means in the runner changes.
• Honest scope: this is a runner-loop change, not a runtime one. Today output extraction and the validation script run after the agent process returns, once per iteration. They move to the turn-end path the steer watcher already has, with the agent idle on stdin. max_iterations stays the cap.
• The sweep leaves the validation loop. It remains for one thing: interrupting a Codex turn mid-flight on a steer in feat(dispatch)!: steer the agent run already in flight on work-item updates #6959, until Close issue #2574: recommendation contradicts ADR 43 and targets relocated code #3159 gives us a proper primitive.

I had filed #6965 this morning as "resume the session on retry". I'll reframe it to the above, with the fresh-process retry kept only as the fallback when a session cannot be continued.

Related ADRs in flight

Two open ADRs decide the platform side of the same question, and the check described above has to line up with them. @ifireball owns both, so flagging for a sanity check of the fit:

• ADR 0098, docs(adr): serialize agent runs and coalesce subsequent events #6909: serialize agent runs and coalesce subsequent events. Preserve the active run, coalesce later events into one pending run, and require each run to reconcile the subject's current state. A validation turn on the live session is the active run reconciling before it ends, not a new run.
• ADR 0099, docs(adr): adopt entity-first harness evaluation #6956: entity-first harness evaluation. One predicate over the resolved entity plus per-harness processing receipts. The pass/fail outcome of the host-side check is the receipt a later evaluation would read.

On the MCP tool

We don't run MCP servers for agents, remote or as a sidecar in the sandbox: it costs context on every turn, VM resources, and a second control surface next to the harness. A plain tool is enough, and it already exists: agents run fullsend-check-output against the schema before they exit (#2777 is the code agent missing that step). The host side then does the schema and outcome checks in the validation script and post-script.

The turn boundary is what the runner can rely on whether or not the agent remembers to call anything, so I'd keep the runner-driven check as the contract and the in-sandbox tool as the agent's early signal.