chore: block merges while Codex reviews are unfinished - #2
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5d372733d0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const codex = {}; | ||
| for (const p of open) { | ||
| const comments = await github.paginate(github.rest.issues.listComments, {...args, issue_number:p.number, per_page:100}); | ||
| const reactions = await github.paginate(github.rest.reactions.listForIssue, {...args, issue_number:p.number, per_page:100}); |
There was a problem hiding this comment.
When Codex acknowledges a review request by reacting to its PR comment, listForIssue cannot see that reaction because it only enumerates reactions attached directly to the PR issue; comment reactions require the issue-comment reactions endpoint. Consequently, the documented legacy reaction-only case produces eyes=[], and even a manual dispatch can publish a successful check while that review remains active if no matching request or summary remains.
AGENTS.md reference: AGENTS.md:L12-L12
Useful? React with 👍 / 👎.
| const codex = {}; | ||
| for (const p of open) { | ||
| const comments = await github.paginate(github.rest.issues.listComments, {...args, issue_number:p.number, per_page:100}); | ||
| const reactions = await github.paginate(github.rest.reactions.listForIssue, {...args, issue_number:p.number, per_page:100}); | ||
| codex[p.number] = codexReview(comments, reactions); |
There was a problem hiding this comment.
For N open PRs, each snapshot() now performs 2N serial comment/reaction requests, while the publication loop calls snapshot() again for each of the roughly N head SHAs. A single reconciliation therefore makes O(N²) GitHub API calls—for example, about 1,000 requests around 20 open PRs before pagination—which can exhaust the workflow token quota or the 10-minute timeout and leave every required Stack policy check failing.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b00f0762ca
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const completedAt = Math.max(0, ...summaries.map(c => Date.parse(c.updated_at) || 0)); | ||
| const requests = comments.filter(c => c.authorizedRequest === true && isReviewRequest(c.body)); | ||
| if (requests.some(c => (Date.parse(c.updated_at || c.created_at) || Infinity) >= completedAt)) | ||
| return {ok:false, message:'A Codex review was requested; waiting for the bot completion summary.'}; |
There was a problem hiding this comment.
When reviews overlap, an older review can complete after a newer @codex request was posted; because completedAt is the maximum timestamp across every summary, that unrelated completion makes the newer request appear satisfied before its review emits a status or reaction. codexReview then returns success, and since reaction creation is not among this workflow's triggers, the successful required check can remain mergeable throughout the newer review. Track completion by review type/request rather than treating any later completed summary as completing every request.
AGENTS.md reference: AGENTS.md:L12-L12
Useful? React with 👍 / 👎.
1 participant
Codex can be reviewing while existing checks are green. Extend the required Stack policy check to block unfinished reviews using authenticated bot summary comments, authorized manual requests, and PR and comment eyes reactions. Completed findings remain advisory.
Reconcile on PR and comment events, with manual dispatch available for legacy reaction-only reviews. Preserve existing branch protection and the installed action pin. The controller reads metadata only; GitHub event delivery still leaves a brief detection window.
Validation: 44 policy/controller tests passed locally, including concurrent reviews, spoofed summaries, permission checks, repeat requests, linear API reads, deleted markers, head updates, separate review types, shared heads, and publication races. Tests are included under
.github/policy-tests/.Depends on: none