Add Prime Harness reliability layer; fix RLM billing-failure retries and Windows console windows - #1593
Conversation
RLM children whose provider stream ends in a terminal error (e.g. a 402 insufficient-balance response) were reported as silent completion instead of failure, and permanent billing failures were retried like transient ones. Surface the terminal assistant error and stop retrying billing failures.
Detached daemon, worker, and shell child processes opened a visible console window on Windows because spawn() defaults windowsHide to false. Set windowsHide on every detached/background spawn call so these processes launch hidden, matching behavior on other platforms.
Brings in two fixes carried on the branch: - Surface permanent recursive child failures (RLM children no longer retry permanent provider billing failures or report them as silent completion). - Hide spawned child process windows on Windows.
|
Hi @cdimurro, thanks for your interest in contributing! This project requires that pull request authors are vouched, and you are not in the list of vouched users. This PR will be closed automatically. See https://github.com/PrimeIntellect-ai/prime-agent/blob/main/CONTRIBUTING.md for more details. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 943213e15b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| shell: [bash, pwsh] | ||
| defaults: | ||
| run: | ||
| shell: ${{ matrix.shell }} |
There was a problem hiding this comment.
The new standalone CI workflow places ${{ matrix.shell }} under jobs.full-suite.defaults.run.shell, but GitHub does not allow contexts or expressions in defaults.run, so the workflow is rejected before any matrix job runs. Set shell on each run step or split the shell variants into separate jobs. See GitHub's jobs.<job_id>.defaults.run syntax.
Useful? React with 👍 / 👎.
|
|
||
| results_dir = harness_dir() / "results" | ||
| results_dir.mkdir(parents=True, exist_ok=True) | ||
| result_path = (results_dir / f"{child_name}.json").resolve() |
There was a problem hiding this comment.
When spawn(..., name=...) receives an absolute name or one containing ../, joining f"{child_name}.json" discards or escapes results_dir, and resolve() merely canonicalizes that escaped path. The child is then instructed to write its result outside artifacts/harness/results, potentially overwriting an unrelated repository or filesystem file; validate names as safe basenames and verify the resolved result remains under results_dir.
Useful? React with 👍 / 👎.
| if (this._isPermanentProviderBillingFailure(message)) { | ||
| return false; |
There was a problem hiding this comment.
This commit changes user-visible coding-agent behavior for terminal RLM billing errors and Windows subprocess windows, but it adds no packages/coding-agent/.changes/*.md fragment. Because releases fold these fragments into the changelog, both fixes will be omitted from release notes unless a coding-agent fragment is added.
AGENTS.md reference: AGENTS.md:L119-L123
Useful? React with 👍 / 👎.
1 participant
Summary
prime-harness/): a reliability-first workflow orchestration harness for Prime Agent — deterministic scientific oracles, persistent provenance (evidence ledger), delegation/budget discipline, an independent cross-harness critic, and bounded graph-ranked repo reconnaissance. Seeprime-harness/README.md.windowsHideon every relevantspawn()call.Test plan
npm run check(biome, tsgo type-check, installer render check, browser smoke check) passes on the merged treevitest --run test/agent-session-recursion.test.ts test/suite/agent-session-retry-events.test.ts— new billing-failure regression tests pass; 2 pre-existing Windows-environment failures (temp-dir cleanup EPERM, an unrelated kernel-depth-cap probe) reproduce identically with and without these changes, confirmed via a baseline worktree comparisonagent/prime-harness-world-classintomaincompleted with zero conflicts (git merge-treeverified before merging)Note
Add Prime Harness layer; fix RLM billing retries and Windows console windows
prime-harnessreliability layer containing new Python skills (evidence_ledger,external_critic,harness_orchestrator,repo_map,sci_verify), CLI tools (verify.py,doctor.py,replay.py,scorecard.py), and comprehensive test suites.AgentSessionin agent-session.ts to throw on terminal provider errors and short-circuit retries via the new_isPermanentProviderBillingFailurehelper when a 402 or billing error is detected.windowsHide: process.platform === "win32", preventing console window flashes on Windows.venvPythonPathhelper in bootstrap.ts to correctly resolve venv Python executable paths on Windows.AgentSessionchild-run completion now throws an error if the child ends withstopReason === "error"instead of proceeding silently.Macroscope summarized 943213e.