Skip to content

fix: terminate owned foreground command groups on cancellation - #767

Draft
nikhilachale wants to merge 16 commits into
stakpak:mainfrom
nikhilachale:fix/foreground-command-cancellation
Draft

nikhilachale wants to merge 16 commits into
stakpak:mainfrom
nikhilachale:fix/foreground-command-cancellation

Conversation

@nikhilachale

@nikhilachale nikhilachale commented Oct 12, 2026 •

Copy link
Copy Markdown

Code changes: +131 -24
Tests: +307 -0
Others: +102 -0

Description

Foreground run_command creates a Unix process group but stock cancellation/timeout kill only the shell leader. Nested shell/sleep processes can keep executing and write after Escape. Own the child through a small drop guard; signal its group while the leader is still unreaped on cancel, timeout and future drop, then bound explicit reaping to five seconds. Read Child::id() instead of retaining a numeric PGID after reaping. Keep the persistent MCP worker and unrelated processes outside this ownership.

The real subsequent-command regression also exposed lost output: the executor stopped at either pipe's EOF, reaped before draining, and used cancellation-unsafe read_line inside select!. Drain both streams using cancellation-safe Lines::next_line before reaping. An exited leader whose descendants still hold pipes open remains owned and cancellable. Normal successful commands are not group-killed; commands that background children while retaining the foreground pipes now wait for pipe EOF (use the existing background task tool for detached work).

Evidence and verification

Observed with stock v0.3.88 (d3eba6f), preserved in AO6550. Stock remains FAIL. This source fix is unreleased.

Nine focused regressions use real ToolContainer/MCP duplex calls and the actual executor: cancellation with/without timeout, timeout, coordinated transport shutdown, execution-future abort independent of token cancellation, leader exit with open pipes, and successful wait/try_wait lifetime controls plus an actual external-reap ECHILD regression. Wait errors indicating lost child ownership disarm both group and direct-child signal authority; fallback killing happens synchronously in the owner, never through a deferred kill-on-drop after ECHILD. Nested shell/sleep start markers and kernel start identities establish execution; delayed-artifact absence and an unrelated live control establish isolation; the same MCP session must execute and return stdout for a subsequent request. Stock executor compilation is required before its cancellation regression must fail.

Least-privilege fork qualification Actions runs focused tests, stock failure, affected-package lint and an independently bounded Linux prerelease artifact build; a separate bounded job runs upstream lint, the complete workspace/feature suites and build. No publication/deployment jobs or provider secrets are used. All routine validation is CI-only by user policy; no local suites or native compilation. Current head62bd883 has a green Linux qualification/artifact job:9/9 regressions, compiled stock negative control, affected lint and release build. Exact Linux artifact SHA-256: 882e7dce06ce64afe76249befff732a5f17b86c5594291526b5302a323965ac2. Full upstream suites are separately pending/unverified; repeated hosted runs outlasted their timeout and the cancelled attempt's log returned404 BlobNotFound. No full-suite pass is claimed.

Earlier source16e63ed artifact SHA84ac1f... passed the unchanged native Escape reproducer, original immediate shell/sleep PID absence, no delayed write, unrelated control and a third turn in the same native identity. Actual AO Electron standalone-shell output and Linux runtime-close/control evidence were captured separately. These are prerelease controls; current62bd883 runtime qualification is ongoing.

Exact current artifact62bd883 passed22 native checks inside the real AO Electron standalone terminal: initial task once/clean leading-dash, correct cwd, private/project rules, preserved AGENTS, three active-to-settled turns, Escape after real shell/sleep start markers, original immediate PID absence, no delayed write, unrelated control heartbeat and same native identity after cancellation/third turn. Native timeout was35s; cancellation preceded expiry. Immutable report, screenshots and curated receipt. This is supplementary standalone-shell proof, not registered AO harness/manager/Kill conformance. Earlier forced-renderer-reload OSC11 composer corruption is preserved as FAIL; the clean fresh terminal control does not erase it. Both runtime shells, hosts, owned Electron and scratch daemon were cleaned with numeric API receipts and identity checks.

Full native upstream suites remain CANCELLED/NOT_VERIFIED after repeated hosted stalls and unavailable logs. This legitimate CI availability gap keeps the native PR draft. AO6550 remains draft/unregistered pending an official reviewed release and separate fresh-only manager/Kill/UI qualification. Stock0.3.88 is unchanged/failed.

Scope

Unix process-group cleanup only; commands deliberately creating another group/session are outside this boundary. Windows preserves direct-child cancellation without a descendant guarantee. Exclusive child ownership is assumed; no global process scanner/reaper is introduced. Visible TUI cancellation still precedes server cleanup acknowledgement, so bounded native process/no-late-side-effect proof remains necessary. No restore/history/default-instruction/permission changes.

@nikhilachale

Copy link
Copy Markdown
Author

Concrete Stakpak native foreground fix is implemented in stakpak/agent767, source 62bd8833600b0f906ff841930433264ece36dbab. Exact CI-built Linux artifact SHA 882e7dce06ce64afe76249befff732a5f17b86c5594291526b5302a323965ac2; it reports0.3.88 but is explicitly unreleased patched source, not repaired stock release.

Current qualification job: 9/9 PASS, compiled stock negative control, affected lint and release build. Full upstream workspace/feature suite: CANCELLED/NOT_VERIFIED after repeated hosted stalls/timeouts and unavailable logs; native PR stays draft. No routine local suites/builds/native compilation ran.

Exact artifact passed22 native checks in the actual AO Electron standalone terminal: initial leading-dash task once in history, real cwd, private/project instructions, AGENTS preservation, initial/second/third active鈫抯ettled turns, observed foreground shell/sleep PID/start witnesses, Escape with immediate physical PID absence, no25-second delayed artifact, unrelated PID/start/heartbeat control, same native identity and third turn after cancellation. Tool timeout35s exceeded the cancellation time. This is supplementary native/runtime evidence; AO6550 remains unregistered/draft, with no manager/agent-session Kill, restore, Chat, handoff, catalog or registered-auth claims.

Immutable detailed report and receipt. All four PNG URLs were downloaded and SHA-256 verified after publication; images inspected for credentials. Backend/UI source fb60bd4097ebd4109bbe7ee68e908f722e6ee03f; serving PID979004/hash b30f2dfa7b1f1c796c5772cb96d7d5c32b4bfb53c39f2c4ff7ca669f2cca8ded before/after. Native identity digest 8082f814ca93f1963292a78cc9d01f18f4ed3fc1fce8ec7b7f8fe7d8fa20d198; runtime handle ptyhost-v1:shellterm-8968a375bddfe0aa, host985380/start82945904. Standalone shells have no managed agent generation. Model rendered GLM-5.3-Flash/provider Zai/profile audit. Capture: real Electron/scrot on owned Xvfb display.

UTC2026-10-12T05:56:59Z, native command executing; PNG SHA 01432b5596ea1d7c7700d454386279154d0d13fcf82a029668c18fee54497786:

Unreleased source62bd883 native foreground command in actual AO standalone terminal

UTC2026-10-12T06:03:07Z, same native identity after Escape and third count3 turn; PNG SHA 526ccf4f6a99cf97bbf45b362b2080fda640c93123083b21005cbf760e54da10:

Same unreleased Stakpak session after cancellation and third turn in actual AO

Retained failure: earlier16e63ed startup after forced renderer reload inserted OSC11 reply text before the prompt (FAIL, UTC05:17:23Z); actual failed screen. Clean fresh launch is a separate passing control; reconnect behavior remains unqualified. Default security/permissions were preserved. Original stock Escape failure and restore-system duplication remain failures; original AO screenshot unavailable because no registered session existed then.

Cleanup: both final shell closes204; shell list200/empty; runtime hosts/native/control PIDs absent; owned Electron main978778 and launch members stopped; scratch daemon979004 shutdown202 and absent; mux helpers stopped and compute lock released. Production625660/ao-self-hosted, profiles/credentials/source/shared git/prior evidence untouched. Runner SHA8887aa2e... unchanged. No merge, release publish, deployment, signup or billing change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant