Skip to content

fix(review): surface the head commit so a resumed session can't mistake a re-review for a repeat - #73

Merged
Mearman merged 1 commit into
mainfrom
fix/review-resume-stale-commit-ground-truth
Sep 17, 2026
Merged

Mearman merged 1 commit into
mainfrom
fix/review-resume-stale-commit-ground-truth

Conversation

@Mearman

@Mearman Mearman commented Sep 17, 2026

Copy link
Copy Markdown
Member

What this fixes

On a long-lived pull request with many commits, review mode's session resumption
(resume_sessions, on by default) carries every prior run's conversation forward as one
growing transcript. Nothing in the composed prompt ever states the current head commit
explicitly, so a resumed session has no mechanical way to check its own memory against
reality. Confirmed twice independently on the same downstream repository, across two
different pull requests: a review got recorded against the PR's true current head commit
(via GitHub's own review metadata), but the review's own written reasoning claimed the
triggering commit was identical to the one it last reviewed, even re-flagging findings that
had already been fixed in the new commit.

Root cause

The "Compose prompt" step's "This run" facts block (action.yml) never surfaces the current
head commit SHA anywhere. Every other run-scoped fact the model needs (repository, mode,
entity number, base branch, severity mode, and so on) is stated explicitly so the model
doesn't have to infer it — the head commit was the one fact missing, and it's exactly the one
a resumed session most needs to check itself against.

The fix

  • Add HEAD_SHA to the "Compose prompt" step's env block, resolved from
    github.event.pull_request.head.sha — the same context the "Decide whether this event
    changed anything worth reviewing" gate step already reads it from.
  • Print it as a new - Head commit: \`fact line in the "This run" section, scoped to review mode (this is PR/commit-scoped information, not relevant to triage or interactive modes), and unconditional onresume_sessions` — always present whether or not
    the session was resumed.
  • Add an explicit instruction to the "Resolving stale threads and reviews on re-review"
    section of the review prompt (prompts/review/base.md): before treating a run as a
    re-review of an already-seen commit, check the "Head commit" fact against what the model
    actually reviewed last (e.g. the commit_id on its most recent review), rather than
    assuming a resumed session's transcript is still on the right commit.

Validation

  • npx turbo run typecheck lint format:check — all pass.
  • npx commitlint --from HEAD~1 --to HEAD — the commit message passes.
  • action.yml's runs.steps isn't directly checked by actionlint (it only understands
    workflow files); validated it anyway using the README's own documented technique — wrapping
    the steps array in a synthetic workflow_call reusable workflow and running actionlint
    against that. Diffed the resulting warning set against the same synthetic wrapper built from
    main: one additional SC2016 (style-level, "expressions don't expand in single quotes")
    note appears, from the new printf fact line following the exact same
    printf -- '- Label: `%s`\n' "$VAR" pattern every other fact line in this block already
    uses and already triggers the identical note for — not a new class of issue.

Not done here

No CHANGELOG.md edit — it's generated by semantic-release from commit history, not hand-maintained.

…ke a re-review for a repeat

A resumed session (resume_sessions: true, the default) carries every prior commit's conversation
forward as one growing transcript, but nothing in the composed prompt states the current head
commit explicitly. On a long-lived pull request the model has no mechanical way to check its own
resumed memory against reality, and can conclude a new commit is the same one it already reviewed.

Add the head commit SHA as its own fact line in the "This run" section for review mode, resolved
from the same pull-request event context the gate step already uses, and always present regardless
of resume_sessions. Also instruct the model, in the re-review section of the review prompt, to
check that fact against what it actually remembers reviewing before treating a run as a repeat.
@Mearman
Mearman marked this pull request as ready for review September 17, 2026 14:22
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review Completed 2026-09-17T14:29:05.220059Z 165711d Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@Mearman
Mearman merged commit 8ddf085 into main Sep 17, 2026
13 checks passed
@Mearman
Mearman deleted the fix/review-resume-stale-commit-ground-truth branch September 17, 2026 14:28

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small, well-scoped fix that does exactly what the description says: adds HEAD_SHA as a review-mode-only fact line, sourced from the same github.event.pull_request.head.sha context the Dependabot gate step already reads, and pairs it with an explicit instruction in prompts/review/base.md to check it before trusting a resumed session's own memory of "already reviewed this." Verified the mechanism actually works end-to-end: this very review's own "This run" facts include Head commit: 165711d75b1ffba39f85238b0015d1b935b5c95a, matching this PR's real head commit, and the new prompt paragraph is present in the instructions I'm following right now.

Checked for regressions: dependabot.yml's own direct-form review call is also pull_request-triggered, so HEAD_SHA resolves there too, not just through the reusable-workflow path. No prior reviews or inline comments exist on this PR, so there's nothing to reconcile from an earlier revision.

No Blockers or Should-fix findings. Two minor observations, neither worth blocking:

  • [stack-default] HEAD_SHA has no override input analogous to pr_number/inputs.pr_number for ENTITY_NUMBER in action.yml, so a hypothetical future workflow_dispatch-driven review with an explicit pr_number override would silently omit the "Head commit" line (the if [ -n "${HEAD_SHA:-}" ] guard degrades gracefully rather than erroring, so this isn't a bug today — review mode has no such trigger currently — just a gap worth remembering if one is ever added).
  • The PR description's own validation section is unusually thorough (typecheck/lint/format, commitlint, and a manual actionlint diff against main for the composite-action steps) — genuinely useful for a reviewer, worth calling out as good practice rather than a finding.

Title and description are already complete and accurate; nothing to correct there. No unreadiness signal, so no draft conversion.

@github-actions

Copy link
Copy Markdown

🎉 This PR is included in version 1.12.2 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This run is triggered against the exact same head commit (165711d75b1ffba39f85238b0015d1b935b5c95a) already carrying an APPROVED review from this same identity, and the PR has since merged (8ddf0853427f461fff78b7dab9c293f8d0885b19). Per the "Head commit" fact this PR itself teaches review mode to check, and per the commit_id on the existing review, this is not a new revision to review — so no fresh review comments are being added, and nothing from the prior review needs dismissing or updating (resolving stale threads only applies to an older commit than the current head, which this isn't).

Independently re-verified the change against the diff and the PR description's own technical claims, and reached the same conclusion as the prior review:

  • HEAD_SHA in action.yml's "Compose prompt" step is sourced from github.event.pull_request.head.sha — the same context examples/claude-review.yml's own gate step already reads it from, as the description states. Confirmed via grep -n "head.sha" action.yml that this is the only occurrence, so no duplicate or conflicting logic.
  • The new - Head commit: fact line is correctly scoped to review mode only and unconditional on resume_sessions, matching the stated intent.
  • The new paragraph in prompts/review/base.md is correctly placed in the "Resolving stale threads and reviews on re-review" section, instructing exactly the check needed to fix the described bug.
  • No Blocker or Should-fix findings. Title and description are complete and accurate; no unreadiness signal, so no draft conversion.

Two minor observations, both already raised by the prior review and neither worth blocking:

  • [stack-default] Nit: HEAD_SHA has no override analogous to pr_number/ENTITY_NUMBER, so a hypothetical future workflow_dispatch-driven review with an explicit pr_number override would silently omit the "Head commit" line (the [ -n "${HEAD_SHA:-}" ] guard degrades gracefully rather than erroring, and review mode has no such trigger today, so this is a gap worth remembering rather than a live bug).
  • The PR description's validation section (typecheck/lint/format, commitlint, a manual actionlint diff against main's synthetic wrapper) is unusually thorough for a 6-line change — worth calling out as good practice, not a finding.

Approving again, consistent with the prior review's own honest state (no Should-fix or Blocker findings) and the "always submit a decisive state" rule, rather than falling back to a non-committal comment merely because the verdict is unchanged.

@github-actions

Copy link
Copy Markdown

🗜️ Headroom context compression

Metric Value
Requests proxied 48
Tokens saved 66608
Aggregate savings 1.3% of all tokens sent
Average per-request compression 1.8%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant