Skip to content

feat(oversee): add --verify for out-of-session independent VERIFY - #2655

Open
carlos-alm wants to merge 1 commit into
mainfrom
feat/oversee-verify-mode
Open

feat(oversee): add --verify for out-of-session independent VERIFY#2655
carlos-alm wants to merge 1 commit into
mainfrom
feat/oversee-verify-mode

Conversation

@carlos-alm

Copy link
Copy Markdown
Contributor

/oversee --verify <#execute-PR> re-runs independent VERIFY on an open execute PR (same verifyPrompt / VERIFY_SCHEMA agent as in-session re-verify). The skill never POSTs pipeline/verify. Exclusive with the other phase flags.

@greptile-apps

greptile-apps Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds an out-of-session /oversee --verify mode that reconstructs task context from an execute PR and launches the existing independent verifier.

  • Adds --verify argument parsing, exclusivity, and routing.
  • Defines execute-PR context recovery, verifier dispatch, result gating, and reporting.
  • Documents the new invocation and persisted phase state.

Confidence Score: 3/5

The PR is not safe to merge until the VERIFY result contract is aligned and the target is proven to be an execute PR.

The new flow consumes fields its verifier schema cannot return, breaking successful re-verification, and allows mutable branch/body metadata to substitute for execute-PR provenance.

Files Needing Attention: .claude/skills/oversee/SKILL.md

Important Files Changed

Filename Overview
.claude/skills/oversee/SKILL.md Adds the complete out-of-session VERIFY flow, but its result gate is incompatible with the selected schema and its target validation does not establish execute-PR provenance.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A["/oversee --verify #PR"] --> B["Route to Verify"]
  B --> C["Validate open non-Plan PR"]
  C --> D["Recover task and Done-when"]
  D --> E["Spawn verifyPrompt with VERIFY_SCHEMA"]
  E --> F["Gate structured result and live head"]
  F --> G["Print VERIFY board and stop"]
Loading

Fix all with Greploop Fix All in Claude Code

Reviews (1): Last reviewed commit: "feat(oversee): add --verify for out-of-s..." | Re-trigger Greptile

4. **Spawn one independent VERIFY agent.** Read `verifyPrompt(t, exec)` and `VERIFY_SCHEMA` from `.claude/workflows/oversee-dispatch.js`. Fill `t.id` / `t.issue` and `exec.executePR = N`. Model: `sonnet`. Isolation: `worktree`. Label: `re-verify:<task-id>`.

**This skill never POSTs `pipeline/verify` and never `gh pr review`.** Only the spawned agent may stamp. Never merge. Never dispatch `oversee-dispatch`.
5. **Gate the result** as `applyVerifyGates` does: `verifiedSha` must be 40-hex; `pass` with `reviewPosted` or `statusPosted` false → fail. Re-read live `headRefOid`; mismatch → STOP and tell the operator to re-run `/oversee --verify #N`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Verifier result contract is incompatible

When the spawned verifier completes, VERIFY_SCHEMA returns only verdict (approve or changes-requested), blocking, and summary, but this gate expects pass, verifiedSha, reviewPosted, and statusPosted. Consequently, the mode either rejects every valid result or bypasses the promised head and posting checks.

Fix in Claude Code

Reached only from Phase: Route when `--verify` is set. `N` is an **open execute PR**, not a `[Plan]` PR.

1. **Refuse `[Plan]` PRs and closed PRs.** Title starting with `[Plan]` → STOP (that is `--execute` / a plan gate, not VERIFY). Closed → STOP. Drafts are OPEN.
2. **Recover task id + Done-when — fail closed.** Snapshot `headRefName`, `body`, `headRefOid`. Prefer `execute/<task-id>` or the collision suffix. Else closing keywords / `Part of #<issue>` → tracking issue. Done-when from the issue body (this repo has no master-roadmap §3). STOP if either is empty.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Execute PR provenance is unenforced

When --verify targets an open non-Plan PR whose branch or body references an issue, this recovery accepts mutable metadata without establishing that the PR descends from the approved plan. The verifier then reviews and can stamp an unrelated PR against that issue's Done-when.

Fix in Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant