Skip to content

feat: guard unregistered Freebuff native qualification - #6553

Draft
nikhilachale wants to merge 4 commits into
OrchestratorInc:mainfrom
nikhilachale:feat/freebuff-native-qualification-20261011
Draft

nikhilachale wants to merge 4 commits into
OrchestratorInc:mainfrom
nikhilachale:feat/freebuff-native-qualification-20261011

Conversation

@nikhilachale

@nikhilachale nikhilachale commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Code changes: +118 -0
Tests: +146 -0
Others: +1399 -0

Binary assets in this source snapshot: 8 (excluded from line counts).

Draft qualification boundary; unregistered and launch/hooks/restore guarded.

Current status

Current source snapshot: 8665c30db42b24d7945f201eab4b2a507ca69706 (base 95005365cabed5e603b3329f10487165dd1053b7). Draft; GitHub mergeability MERGEABLE, merge state BLOCKED. Exact-head CI: all 20/20 reported checks successful. CI run 1 · CI run 2 · CI run 3. Green source CI does not establish current native/UI runtime proof or full integration readiness.

This fix created only clean isolated source worktrees and temporary review reports; it created no AO/native sessions, provider profiles or owned runtime processes. No local build, gofmt, test, fixture, provider or UI run was performed.

Historical runtime build: No registered AO runtime build is qualified. Historical stock Freebuff0.2.26 failed independent instructions. Supplementary native/prerelease standalone-terminal capture used AO/UI fb60bd4097ebd4109bbe7ee68e908f722e6ee03f and upstream source 5712d04bc916125d17da46548874a0388dcc526e (0.0.0-ci, executable SHA256 5071e80f5eec7dd7a80d055e0ca0641b17470c7da44067d5ba8f032af9108822). Report · evidence index, original receipts, image/provenance hashes. Receipt SHAs/bytes are unchanged. Linux/VPS proof is not Windows/macOS native proof. Runner SHA256 8887aa2e7101b9e6d3950bfca08ffddf2e1e6c1c3f1d030c8dec2676443aecea remains historical where reported; manual/standalone proof is not a strict runner PASS.

Changes since runtime proof: Source remains an intentionally guarded draft/unregistered qualification boundary. Evidence relocation did not admit stock Freebuff or rerun native conformance.

The matrix separates current verification from recorded historical passes; NOT_RUN does not erase those passes. Excluded interfaces/manual model input are deliberate supported-scope decisions.

Area Supported scope Verification Proof or reason
Registration and UI Draft qualification boundary; unregistered and launch/hooks/restore guarded. BLOCKED Intentional unregistered qualification hold; registration is not missing work.
Models No product model picker advertised; native BYOK/model controls only. N/A No product capability advertised while unregistered.
Setup Qualification prerequisites only; no product setup plan advertised. N/A Qualification-only scope has no product setup plan.
Readiness Admission is deliberately refused until released native prerequisites pass. BLOCKED Released-native prerequisites block admission.
Identity and logo No product avatar/label advertised while unregistered. N/A No production avatar advertised.
TUI and Chat No registered TUI or Chat. Standalone terminal evidence is supplementary native proof. BLOCKED Standalone native/prerelease captures are not registered AO proof.
Permissions and instructions Stock private ordinary-task instruction consumption failed; patched prerelease proof is separate. BLOCKED Stock failure and prerelease controls remain distinct.
Lifecycle Stock exact restore remains unqualified; no manager Kill/restore proof. BLOCKED Registered AO controls NOT_RUN; retained stock failures unresolved.
Evidence Pinned historical source/native/runner/UI receipts, attempts, images and provenance; missing originals disclosed. NOT_RUN Historical evidence retained; current native/UI proof NOT_RUN. Reproduction-input limitations below remain explicit.
CI and handoff Exact source checks/merge snapshot; incomplete live qualification and historical cleanup. BLOCKED Exact-head CI all 20/20 reported checks successful; mergeability MERGEABLE. Released-native admission remains blocked; draft/unregistered.

Historical implementation and evidence

Released0.2.26 native BYOK/edit/follow-up/activity and witnessed Escape/reaping/no-late-effect controls passed historically, but an ordinary fresh task did not create the independently required private receipt. Prompted file reading passed only as a separate assisted control. No supported released additive input was found; public-source prerelease proof does not repair stock0.2.26 or qualify renewed restore instructions. The AO adapter hard-refuses launch/hooks/restore; issue6542 stays open and historical closed/unmerged Codebuff1826 is distinct. Historical source fcdc3db was 20/20 green; evidence aa0a700 was 5/5 green/150 fixtures, preserving the first 149/150 failure.

historical receipt 1 · historical receipt 2 · historical receipt 3

Reproduction and retained-original limitations

The pinned reports/JSON retain action/result, file/token effects, process identity, cancellation, native-history continuity and capture assertions. A complete sanitized executable reproduction bundle is not established for every manual helper: exact helper bytes/hash and invocation, applicable local-contract bytes/hash with probe argv and interrupt/readiness/draft-cleanup settings, and complete native/AO fingerprints are unavailable where not recorded in the linked receipts; recoverability of those original inputs is unknown. No missing command, contract, hash or result is invented. Later reproductions must use a new attempt and cannot replace original failed/missing evidence.

Recorded owned-process/session/daemon/Electron cleanup is historical. Source worktrees, native profiles/history, dirty scratch workspaces, binaries and raw logs were reported retained privately; a complete current inventory/location/hash and recoverability of every original class are unavailable in the public record. Current recoverability is unknown; this reporting fix reconstructs no missing input and silently substitutes none. Share sanitized artifacts only; exclude credential stores, raw DBs and provider-profile directories. Preserve the retained failures and original screenshots.

@github-actions

Copy link
Copy Markdown
Contributor

🏆 Review leaderboard

Oct 4–11, 2026 · UTC

Rank Reviewer PRs reviewed Review rounds PR comments
🥇 @illegalcall @illegalcall 31 51 8
🥈 @ronishrohan @ronishrohan 16 19 11
🥉 @nikhilachale @nikhilachale 12 25 3
4 @Prasad-D-Ware @Prasad-D-Ware 12 21 0
5 @Vaibhaav-Tiwari @Vaibhaav-Tiwari 8 11 3
6 @axisrow @axisrow 8 10 7
7 @mohakchakraborty2004 @mohakchakraborty2004 7 7 1
8 @codebanditssss @codebanditssss 6 10 2
9 @Pritom14 @Pritom14 5 5 4
10 @Rishet11 @Rishet11 4 10 2

Ranked by distinct external PRs reviewed, then review rounds, then PR comments. Self-activity and bot activity are excluded.

Show 5 more reviewers
Rank Reviewer PRs reviewed Review rounds PR comments
11 @Annieeeee11 @Annieeeee11 3 6 1
12 @neversettle17-101 @neversettle17-101 3 4 0
13 @harshitsinghbhandari @harshitsinghbhandari 2 4 0
14 @whoisasx @whoisasx 2 2 0
15 @AgentWrapper @AgentWrapper 1 3 0

@nikhilachale

nikhilachale commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

Remaining batch handoff at immutable evidence 42803b62ab42644fcf2dd845d913a24cc8012275: aggregate candidate report, Freebuff qualification report.

Freebuff0.2.26: authorized native BYOK validation/edit/follow-up/activity/executing-child Escape cleanup controls passed, but independent private standing instructions failed on an ordinary fresh task. PR6553 remains draft and unregistered. AO UI screenshot unavailable because no AO session was admitted; no substitute screenshot or integration success claim. Blackbox/Warp/Rovo/Hermes ACP prerequisites remain explicit in the report. Completed Docker Agent/Mastra/CodeBuddy reports and their real AO images are preserved. Owned native processes were reaped; no production/merge/release/deployment changes.

@nikhilachale

nikhilachale commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

Final batch evidence head f9da74d5938484d3205da63fe88aa6e0ed8c87d5: 5/5 GitHub Actions checks green, including 150/150 fixtures. Runner SHA256 8887aa2e7101b9e6d3950bfca08ffddf2e1e6c1c3f1d030c8dec2676443aecea is unchanged. No local routine suites were executed.

Final candidate report · Freebuff native hold

Current source heads: Docker Agent #6540, Mastra #6541 and CodeBuddy #6546 each23/23 green with actual AO Electron screenshot evidence. Stakpak #6550 and Pool #6551 each20/20 green but draft/unregistered for native external blockers. Freebuff #6553 source 9fbb97e559c245633554ee6545836af03f7348d7 has21/21 green but stays draft/unregistered: an ordinary fresh task did not consume the private standing instructions; passing prompted-context/edit/follow-up/executing-child-cancel controls remain separate. No AO screenshot is claimed for any hold.

Blackbox release metadata and official alternative endpoints still404; Warp private instructions/auth, Rovo site/token/credits and Hermes ACP instruction delivery/preservation remain unqualified. Existing Hermes TUI #6536 and OpenJarvis #6537 are preserved, each23/23 current-head checks green. Polygraph is excluded as a meta-orchestrator; connector scope is separate. Pool's alternate DeepSeek credential presence check found none in the authorized environment/helpers/50 native and audit configs; no secret was printed or imported.

Published image re-download verification matched all6 PNG hashes, and7 earlier integration comments were read back; current reports and this comment were read back too. Source worktrees, native profiles and prior evidence are preserved. Owned Freebuff TUI/broker descendants are absent, protected PID625660 remains, production untouched. No merge/release/deploy.

@i-trytoohard i-trytoohard added this to the Agents & orchestration milestone Oct 11, 2026
@i-trytoohard i-trytoohard added comp/daemon Go daemon, process lifecycle, and backend control plane. enhancement New feature or request labels Oct 11, 2026
@nikhilachale

nikhilachale commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

Freebuff recovery source fcdc3db2099c20854647b59173200a7524bb32fe; evidence 12601e986650596de963ca71325887dec4bf7645.

Recovery report · source diagnosis · unapplied upstream patch.

Independently confirmed the released binary root-only instruction filter and retained native-005 system prompt: root rules present, private nested rules excluded, ordinary artifact present and private receipt absent. All original controls/failures are retained. Alternatives inspected include CLI/settings/profile scope, custom-agent resolver, internal transport and on-demand skills. Stock0.2.26 exposes no supported private additive input preserving defaults.

The patch provides a concrete proposed bounded private-file loader and additive SDK transport, with unexecuted loader regressions and explicit native acceptance criteria. It was not applied to native code or submitted upstream; its env variable is not a stock feature. Remaining prerequisite is an official released route, followed by native and actual AO proof. PR6553 stays draft/unregistered. AO UI screenshot: unavailable because mandatory standing-instruction admission failed; AO lifecycle/Kill/UI remain NOT_RUN. No local routine tests/builds or new native turns ran. Current-head CI is pending.

@nikhilachale

nikhilachale commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

Freebuff recovery final receipt.

Source PR6553 fcdc3db2099c20854647b59173200a7524bb32fe: 20/20 green exact-head checks. Evidence PR6509 aa0a700d5df4d807aa09473ff7b73609777914cd: 5/5 green, including 150/150 audit fixtures. Passing fixtures; first149/150 failure retained. The short-window fixture now waits for its actual child marker;80ms/120ms bounds and production runner remain unchanged. Installed runner SHA8887aa2e remains unchanged.

Immutable recovery report · precise source diagnosis · unapplied source patch/regression proposal. Published report download/hash and PR bodies were read back and verified.

Remaining external prerequisite: stock Freebuff0.2.26 filters nested private instructions out of its system prompt and exposes no supported additive per-session input preserving native defaults/user rules. The proposed env input does not exist in this release and was not executed or published upstream. Need an official released route, followed by independent native instruction/cancellation proof and actual AO lifecycle/UI qualification. No duplicate PR, production registration or gate weakening.

PR6553 remains draft/unregistered. AO runtime/Kill/UI: NOT_RUN. AO PNG: unavailable because the mandatory independent instruction gate failed; no synthetic substitute or integration success claim. All original negative evidence/controls are preserved. No local routine suites, builds, new native turns, AO sessions or cleanup kills were performed. Both owned source checkouts are clean; production/native profiles/protected audit process were untouched.

@nikhilachale

nikhilachale commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator Author

Failure report and fix implementation started (2026-10-12).

Stock Freebuff 0.2.26 discovered the nested private instruction file but did not consume it as standing context on an ordinary task. The released system-prompt filter includes only root agents.md/claude.md; the private nested key was absent from effective system context. Explicitly prompting a file read was only a control, not a passing instruction-delivery test. Preserved diagnosis and proposal.

The previous patch was unapplied. Actual native source implementation is now scheduled in the active durable VPS fix worker, after Stakpak. It will add an explicit additive context input preserving native/project/user instructions, review fresh-session versus cached-context behavior, and cover real CLI/client/model-request composition in CI. The helper-only proposal is not treated as a complete fix.

A concrete upstream fix PR and exact-head CI/native results will be linked here. Public-source prerelease artifacts will be labelled separately from the private npm release; no claim that stock 0.2.26 is repaired. AO screenshots remain unavailable until the fixed native path is qualified and the AO integration is exercised. No Mac changes/tests/builds, no merge or release.

@nikhilachale

nikhilachale commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator Author

Implemented and published native Freebuff fix PR1505. This is actual public source and regression tests, not the previous unapplied patch. Exact head5712d04bc916125d17da46548874a0388dcc526e; CI-built Linux0.0.0-ci binary SHA 5071e80f5eec7dd7a80d055e0ca0641b17470c7da44067d5ba8f032af9108822. No stock npm0.2.26 repair or registered AO harness claim.

The explicit creation-time FREEBUFF_APPEND_INSTRUCTIONS_FILE/SDK appendInstructions snapshot preserves native/project/user defaults. Existing constructor forwarding works. SDK validates type/Unicode/bounds; POSIX file errors are fail-closed and sanitized. Subsequent turns retain the snapshot; changed/retrofitted history fails before inference. Cached compaction prompts are not falsely refreshed. Ordinary native history/traces retain context; custom agents without the knowledge placeholder and Windows file input are outside qualified scope.

Exact-head fork CI:5/5 focused regressions PASS, real constructor→run→native prompt→provider composition, original-run.ts negative control PASS, Linux runtime build and binary smoke PASS. Full SDK declaration checkFAIL, independently reproduced on unmodified main; final CI step explicitly preserves the failure. Upstream Public CIACTION_REQUIRED needs maintainer approval. Both native1505 and AO6553 remain draft. No routine local suites/install/builds were run; CI owns validation.

Actual exact-head native proof ran in the real AO Electron standalone terminal on the VPS: ordinary leading-dash task once, workspace edit, independent private/project receipts, AGENTS preservation, second turn after deleting the private receipt and changing the source file, retained creation snapshot, armed Escape with executing PID/kernel witness and no delayed effect, same native identity and third turn, then independent AO terminal close204. Escape5/5 and close7/7 controlsPASS. This is not registered manager/session Kill or native restore/Chat/auth/catalog conformance.

Immutable detailed report and curated receipts. All8 image URLs downloaded and SHAverified after publication, inspected for credentials. AO backend/UI fb60bd4097ebd4109bbe7ee68e908f722e6ee03f, clean VCS; actual servingPID994173/kernel83308714/executable SHA b30f2dfa7b1f1c796c5772cb96d7d5c32b4bfb53c39f2c4ff7ca669f2cca8ded before/after capture. Runfile ~/.ao/audits/native-harness-fixes-20261012/reports/codebuff/ao-shell-001/running.json. Runtime handleptyhost-v1:shellterm-46b259221d611db5, native leader997948/start83420715, modelGLM-5.3-Flash/Z.ai. Native identity digest 98a00a64ba65da215f59b2d0d2daa7490a0a07bb64fd3955a47c6d85489b4723; standalone shells have no managed agent launch generation. Images are scrot captures on the actual owned Electron/Xvfb display.

UTC2026-10-12T07:03:36Z, initial ordinary task settled, PNG SHA 0863bf0efda301a34b1ff1ea94c5aa06108d1a7261a26a76e8301c4acf8a44b3:

Unreleased5712d04 independent private/project instructions in actual AO standalone terminal

UTC2026-10-12T07:13:39Z, same native identity and third turn after Escape, PNG SHA bfdb7b8bf171dbd44e9fc02431e0468573b5b5747487cb87b4c23002e0b2e38b:

Unreleased5712d04 same native conversation after cancellation and continuation in real AO

Escape childPID999673/kernel83474776, marker2026-10-12T07:09:55.107636+00:00, Escape2026-10-12T07:09:55.948115+00:00, absentwithin5s/after25s, no delayed artifact, unrelated controlPID994833/start83338796/heartbeat continued. Independent close204 at2026-10-12T07:15:45.615164+00:00 reaped native leader997948 and child1000643, no delayed artifact. Actual pre-Escape execution screen and pre-close execution screen have full capture records in the report.

Failures retained separately: stock nested knowledge exclusion; nonUUID audit fixture before provider; automated evidence driver143 with origin unestablished; grouped paste left an exact owned draft; secondary-before-primary color detection; and manual25s cancel window missed, original late artifact preserved (failed screen). Later armed control kept the same delay/timeout/absence assertions. No global strict auditPASS inferred. Original stock AO screenshot unavailable because admission never created a session.

Cleanup: native/runtime/control closes204, list200/empty, daemon shutdown202; exact owned native/control/PTY hosts and Electron main993952 stopped, mux helpers stopped, compute lock released. Production625660/ao-self-hosted and source/sharedgit/profiles/credentials/prior evidence untouched. Only a verified regenerable old CI artifact was reclaimed after its native PID was absent. Runner SHA8887aa2e... unchanged; current6509 evidence-head CI is tracked separately. No merge, release, deployment, signup or billing change.

Final evidence CI: exact6509 head384c80e5d483fb7edd345148c9ca0cbdbf53fc17 is 5/5SUCCESS, including 150/150fixtures (fixture run, frontend/mac-helper run). Runner SHA8887aa2e... remains unchanged. AO6553 headfcdc3db remains20/20green and draft/unregistered. Native1505 full SDK declarations stillFAIL and upstream Public CIACTION_REQUIRED; no native full-green/released-support claim. Stage result is implemented/native-qualified but blocked on those external prerequisites.

@nikhilachale

Copy link
Copy Markdown
Collaborator Author

Harness-specific reports, JSON receipts, screenshots and provenance now live on this integration PR branch, rather than the skill PR #6509.

Evidence index with embedded screenshots and JSON links.

These screenshots show native/prerelease controls in AO standalone terminals. The harness remains gated/unregistered; this is NOT registered AO integration or manager Kill/restore proof.

This is evidence relocation only. Existing tested source/native hashes and failures are preserved; the documentation commit is not a fresh runtime pass. Applicable checks run in PR CI.

Recorded harness evidence — see scope above

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/daemon Go daemon, process lifecycle, and backend control plane. enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants