Skip to content

chore(ci): cut Codex review cost via path filters, hash skip, draft skip#345

Open
zsculac wants to merge 3 commits intomainfrom
chore/codex-review-cost-cuts
Open

chore(ci): cut Codex review cost via path filters, hash skip, draft skip#345
zsculac wants to merge 3 commits intomainfrom
chore/codex-review-cost-cuts

Conversation

@zsculac
Copy link
Copy Markdown
Contributor

@zsculac zsculac commented Apr 30, 2026

Summary

Codex PR review was running at gpt-5.4 / effort: high on every push, including drafts and noise-only pushes. Cost was unsustainable. This PR ships options A and B from the cost-reduction brainstorm:

  • Effort drop: highmedium
  • Draft skip: drafts no longer trigger review; first run happens on ready_for_review (or non-draft opened)
  • Path filtering: paths-ignore on the trigger plus a matching in-job filter so the model only sees real code (lockfile, snapshots, dist, turbo cache, node_modules, CHANGELOGs, logs, LICENSE, knip.json, top-level data/snapshots all excluded)
  • Hash-based skip: filtered diff is SHA-256'd; if a prior review or skip-comment already carries the same hash marker, the run silently no-ops — kills cost from rebases, branch syncs, and re-runs of identical diffs
  • Size cap: filtered diffs over 5,000 lines get a single comment suggesting a split; the comment embeds the hash so subsequent re-pushes of the same large diff stay silent too

A follow-up PR will add the local pre-push hook (option C from the brainstorm).

Verification

  • actionlint passes clean
  • Traced through scenarios: first review, rebase no-change, oversize first push and oversize re-push, mixed code+lockfile push, draft → ready_for_review

Test plan

  • First push to a new PR triggers a review at medium effort
  • A second push that doesn't change the filtered diff (e.g. rebase) does not consume Codex
  • A push touching only pnpm-lock.yaml does not trigger the workflow
  • A push touching code + pnpm-lock.yaml triggers, but the model's diff excludes the lockfile
  • A PR with a >5,000-line filtered diff posts the size-cap comment and skips the model call
  • A draft PR does not trigger review until marked ready

🤖 Generated with Claude Code

Codex PR review was running at gpt-5.4 / effort high on every push,
including drafts and noise-only pushes (lockfile, snapshots, build
artifacts). Cost was unsustainable.

- Drop reasoning effort from high to medium
- Skip draft PRs entirely; trigger on ready_for_review for the first non-draft run
- Add paths-ignore on the trigger and a matching in-job diff filter so the
  model only sees real code changes
- Compute SHA-256 of the filtered diff and skip silently when a prior review
  or skip-comment already carries the same hash marker (kills rebase/sync churn)
- Cap reviewable diffs at 5,000 lines; post a single comment suggesting a split
  and embed the hash so subsequent re-pushes of the same diff stay silent

Verified with actionlint and traced through normal/rebase/oversize/mixed-push
scenarios.
Comment thread .github/workflows/codex-review.yml
A JSON-parse failure is transient (model hiccup or codex-action error),
but the failure review previously embedded the diff-hash marker. The next
run on the same diff would then match the marker and silently skip,
turning a transient failure into a permanent one.

Drop the marker from the parse-failure path only. All other code paths
(successful review, no-issues-found, all-comments-dropped, size-cap
notice) still embed the marker so genuine successes and permanent skips
remain idempotent across re-pushes.
Comment thread .github/workflows/codex-review.yml Outdated
…-of-diff

If Codex returns comments but every one targets lines outside the right-side
diff, that is almost always a line-mapping failure rather than a real
"no issues" signal. Embedding the hash marker in that case turns a
transient model failure into a permanent skip for the same patch.

Drop the marker on the all-dropped path; only the genuine no-findings
case (zero comments returned) and successful inline reviews still stamp
the marker, so re-pushes of identical diffs only short-circuit when the
prior run actually produced a usable review.
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex review completed — no issues found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant