feat(v1): add trainable Prime Agent harness - #2355
Conversation
6f9ecc5 to
f053b5b
Compare
2f63e73 to
e46f3e2
Compare
f053b5b to
2eed65b
Compare
|
Added the generic Prime Agent ACP lifecycle consumer onto this PR at
Paired producer contract: Prime Agent #1494 at Validation: frozen Ruff/format and |
ApprovabilityVerdict: Not approved Macroscope's review found this PR not approvable — This PR adds a new Prime Agent harness with substantial new runtime behavior: lifecycle state management, terminal quiescence coordination, and modified answer resolution. The scope of new async logic and behavioral changes warrants human review. You can add or adjust custom eligibility rules. Learn more. |
There was a problem hiding this comment.
can remove this and instead add prime agent into e2e test suite
| client.terminal_quiescence is not None | ||
| and client.terminal_quiescence["outcome"] == "error" | ||
| ): | ||
| raise RuntimeError("Prime Agent reported a terminal lifecycle error") |
There was a problem hiding this comment.
cant have prime agent specific code in a generic acp runner
There was a problem hiding this comment.
feels like a lot of addition to the acp runner but dont know this part of the code super well so will let @hallerite judge
There was a problem hiding this comment.
yes feels excessive. I will take a deeper look.
| autonomous: bool = True | ||
| """Run Prime Agent in autonomous mode unless an evaluation opts out.""" | ||
|
|
||
| require_terminal_quiescence: bool = False |
There was a problem hiding this comment.
what is this? does not seem like smth users should set?
There was a problem hiding this comment.
autonomous mode forwards the budget given by verifiers so it is probably ok to only be true instead of user opt-out
Summary
Why
Prime Agent's native model API uses streaming Chat Completions. The harness itself can stay small because the generic renderer-backed streaming bridge now lives in the parent PR, #2369. This PR contains only the Prime Agent integration.
With Prime Agent #1378, ACP-provided MCP tools are translated into session-scoped Python programs by Prime Agent itself. Verifiers supplies the environment's MCP URLs while Prime Agent owns its native tool representation.
Stack and dependency
0.7.3; align the pin with the release containing chore(deps): bump gitpython from 3.1.47 to 3.1.50 in /packages/verifiers-rl #1378 before mergerequire_terminal_quiescence=falseby default), preserving compatibility with0.7.3Validation
uv run --frozen ty check verifierspasseduv run pytest tests/v1 -m 'not e2e'— 70 passedThe lifecycle commits were added append-only. Local hooks that invoke
uv --lockedwere skipped because the existing branch lockfile is stale; the equivalent direct frozen Ruff, formatting, type, and focused test commands above passed.Note
Add trainable Prime Agent harness with ACP lifecycle enforcement
PrimeAgentHarnessandPrimeAgentHarnessConfigin harness.py, which installs Prime Agent, prepares per-trace state directories, emitsmodels.json, and launches it in ACP mode with configurable autonomous/skills flagsVerifiersACPClientnow tracks lifecycle metadata per turn, filters visible replies to current-turn answer chunks, and exposesstop_reason,response_boundary, andterminal_quiescence;promptreturns a structured dict instead of a plain string and enforces correlated lifecycle semantics when configuredlifecycle_meta_namespacetoACPConfigand_record_lifecycle_statusin acp/init.py so lifecycle outcomes are recorded onTrace.infowithout affecting rewards; successful autonomous completions populateacp_answer_fallbackread_answer_file_or_last_replyin score.py to prefer the ACP-recorded fallback reply overlast_replypromptin runner.py now returns a dict (was a string);ACPSession.runandserve_streampropagate this structured type. In-tree callers are updated; out-of-tree consumers expecting a string return will breakChanges since #2355 opened
Macroscope summarized 8a869eb.
Note
Cursor Bugbot is generating a summary for commit 8a869eb. Configure here.