Skip to content

Document Codex review workflow#3

Merged
chrisjcthomas merged 8 commits into
mainfrom
codex/codex-review-setup
Mar 22, 2026
Merged

Document Codex review workflow#3
chrisjcthomas merged 8 commits into
mainfrom
codex/codex-review-setup

Conversation

@chrisjcthomas

@chrisjcthomas chrisjcthomas commented Mar 21, 2026

Copy link
Copy Markdown
Owner

Summary

  • document how Codex review is enabled and requested for this repository
  • add a Codex review section to the PR template
  • tighten AGENTS review guidance for repo-specific P1 review priorities

Validation

  • npm run validate

Notes

  • after opening this PR, request a live Codex review with @codex review to verify the GitHub integration

Open with Devin

Summary by CodeRabbit

  • Documentation
    • Added Codex review setup and usage instructions; clarified review priority rules and when to request reviews or enable automatic reviews; reclassified certain docs/release issues as P1.
  • Chores
    • Added Codex-related checklist items to the PR template; made CI more efficient by skipping packaged/visual steps when irrelevant; bumped core workflow step versions.
  • Tests
    • Relaxed visual snapshot tolerances and added extra visibility/installed-app checks in packaged visual tests.

@chrisjcthomas

Copy link
Copy Markdown
Owner Author

@codex review

@coderabbitai

coderabbitai Bot commented Mar 21, 2026

Copy link
Copy Markdown

Warning

Rate limit exceeded

@chrisjcthomas has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 15 minutes and 36 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d6e36730-eda3-4a3f-8714-fd7a52c37a6a

📥 Commits

Reviewing files that changed from the base of the PR and between 44293e5 and 61d3a12.

📒 Files selected for processing (2)
  • src/renderer/src/App.tsx
  • tests/e2e/visual.packaged.spec.ts
📝 Walkthrough

Walkthrough

Adds Codex review guidance (PR template, README, AGENTS.md), introduces a CI changes job and gates packaged/visual jobs when packaging-related files aren't touched, bumps several GitHub Action majors, and relaxes/extends visual E2E screenshot thresholds and assertions.

Changes

Cohort / File(s) Summary
PR Template
\.github/pull_request_template.md
Added a "Codex review" section with two checklist items: whether @codex review was requested/automatic reviews enabled and whether extra review focus was applied.
Docs & Review Policy
AGENTS.md, README.md
Clarified Codex review priority (P0/P1), reclassified certain release/docs failures as P1, documented two-layer Codex setup, @codex review usage, and optional automatic reviews.
CI: Conditional Changes Detection
\.github/workflows/ci.yml
Added changes job to diff PR files and set needs_packaged output; made packaged-smoke and visual-baselines depend on it and gate steps to skip when packaged changes absent; bumped action majors.
CI: Release Workflow Versions
\.github/workflows/release.yml
Upgraded core action major versions (actions/checkout, actions/setup-node, actions/upload-artifact) with no logic changes.
E2E Visual Test Adjustments
tests/e2e/visual.packaged.spec.ts
Increased maxDiffPixelRatio for certain screenshots, added masking for installed-apps, and expanded queue/populated assertions to include additional APKs and exact installed-app entries.

Sequence Diagram(s)

sequenceDiagram
    rect rgba(240, 248, 255, 0.5)
    participant Dev as PR Author
    participant GH as GitHub (PR + Workflows)
    participant Codex as Codex Review
    participant CI as CI Jobs
    participant Artifacts as Visual Tests
    end

    Dev->>GH: Open PR / comment "@codex review" (optional)
    GH->>Codex: Trigger review request (if enabled/asked)
    Codex-->>GH: Review findings (P0/P1 flags)
    GH->>CI: Start workflows (ci.yml)
    CI->>CI: Run `changes` job (diff file paths)
    alt packaging-related files detected
      CI->>CI: set needs_packaged = true
      CI->>Artifacts: run packaged-smoke & visual-baselines
    else no packaging changes
      CI->>CI: set needs_packaged = false
      CI->>Artifacts: skip packaged/visual steps
    end
    Artifacts-->>CI: report visual test results
    CI-->>GH: Workflow status
    GH-->>Dev: PR checks + Codex feedback
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Poem

🐇 I nudged a checklist, tweaked a job's small art,
Codex listens when a review comment starts.
Screenshots softened, baselines now play nice,
Workflows skip the steps that cost extra slice.
A rabbit hops off—CI purrs, the repo’s smart.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main objective: documenting the Codex review workflow as evidenced by README, PR template, and AGENTS.md updates.
Description check ✅ Passed The description provides a clear summary and validation steps, but deviates from the repository template by omitting UI evidence and artifact testing sections.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/codex-review-setup

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist

Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the repository's documentation and workflow by integrating and clarifying the use of Codex for code reviews. It provides comprehensive instructions for setting up and utilizing Codex, updates the PR template to include a Codex review checklist, and refines the AGENTS file to better align automated review priorities with critical repository concerns.

Highlights

  • Codex Review Documentation: Documented how Codex review is enabled and requested for this repository, providing clear steps for setup and usage.
  • PR Template Update: Added a dedicated 'Codex review' section to the pull request template to guide authors on requesting reviews.
  • AGENTS Review Guidance: Tightened AGENTS review guidance by defining specific P1 review priorities related to installation, documentation, and how Codex reviews should interpret these priorities.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

gemini-code-assist[bot]

This comment was marked as resolved.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

devin-ai-integration[bot]

This comment was marked as resolved.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
.github/workflows/ci.yml (1)

65-82: Conditional execution pattern works but is verbose.

The repeated if conditions on each step achieve the goal but add maintenance overhead. Consider using a job-level if condition or a reusable workflow for cleaner structure. However, this approach does ensure the job always "passes" (important for branch protection) while skipping the actual work.

💡 Alternative: Use job-level condition with a separate always-pass job
  packaged-smoke:
    runs-on: windows-latest
    needs: [changes, validate]
    if: github.event_name != 'pull_request' || needs.changes.outputs.needs_packaged == 'true'
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v6
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: npm run smoke:packaged

  packaged-smoke-skip:
    runs-on: windows-latest
    needs: changes
    if: github.event_name == 'pull_request' && needs.changes.outputs.needs_packaged != 'true'
    steps:
      - run: echo "Skipped for doc-only PR"

This requires both jobs in branch protection rules but is cleaner per-job.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci.yml around lines 65 - 82, The packaged-smoke job
repeats the same if condition on every step which is verbose; change to a
job-level conditional by moving the condition to the job declaration for
packaged-smoke (the job named "packaged-smoke") and remove per-step ifs, and add
a lightweight alternate job (e.g., "packaged-smoke-skip") that runs when
github.event_name == 'pull_request' && needs.changes.outputs.needs_packaged !=
'true' to echo a skip message so branch protection still sees a passing job;
update steps under packaged-smoke to remove the repeated ifs (checkout,
setup-node, npm ci, npm run smoke:packaged) and ensure the two jobs reflect the
same needs: [changes, validate] / needs: changes respectively.
tests/e2e/visual.packaged.spec.ts (1)

9-12: Significant relaxation of visual diff threshold for queue screenshots.

The maxDiffPixelRatio is increased 4× (from 0.025 to 0.1), allowing up to 10% pixel difference before failure. This substantially reduces the sensitivity of visual regression detection for the queue-populated view.

If this is to accommodate expected minor variations (e.g., dynamic content), consider documenting the rationale in a comment. Otherwise, this threshold may mask meaningful visual regressions.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/e2e/visual.packaged.spec.ts` around lines 9 - 12, The change raises the
visual-diff tolerance in queueScreenshotOptions by setting maxDiffPixelRatio to
0.1 which weakens visual regression sensitivity; either revert maxDiffPixelRatio
back to the original 0.025 in queueScreenshotOptions (keep the spread of
screenshotOptions) or, if the increase is intentional, add a concise inline
comment next to queueScreenshotOptions explaining the exact reason (e.g.,
expected dynamic content in the queue view) and consider a narrower fix such as
masking dynamic regions or adjusting the per-test threshold only where necessary
instead of globally raising maxDiffPixelRatio.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In @.github/workflows/ci.yml:
- Around line 65-82: The packaged-smoke job repeats the same if condition on
every step which is verbose; change to a job-level conditional by moving the
condition to the job declaration for packaged-smoke (the job named
"packaged-smoke") and remove per-step ifs, and add a lightweight alternate job
(e.g., "packaged-smoke-skip") that runs when github.event_name == 'pull_request'
&& needs.changes.outputs.needs_packaged != 'true' to echo a skip message so
branch protection still sees a passing job; update steps under packaged-smoke to
remove the repeated ifs (checkout, setup-node, npm ci, npm run smoke:packaged)
and ensure the two jobs reflect the same needs: [changes, validate] / needs:
changes respectively.

In `@tests/e2e/visual.packaged.spec.ts`:
- Around line 9-12: The change raises the visual-diff tolerance in
queueScreenshotOptions by setting maxDiffPixelRatio to 0.1 which weakens visual
regression sensitivity; either revert maxDiffPixelRatio back to the original
0.025 in queueScreenshotOptions (keep the spread of screenshotOptions) or, if
the increase is intentional, add a concise inline comment next to
queueScreenshotOptions explaining the exact reason (e.g., expected dynamic
content in the queue view) and consider a narrower fix such as masking dynamic
regions or adjusting the per-test threshold only where necessary instead of
globally raising maxDiffPixelRatio.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b0c476e2-ec9a-4e20-b37b-c4d94b83b55f

📥 Commits

Reviewing files that changed from the base of the PR and between 3d103f6 and 9bcf12e.

📒 Files selected for processing (5)
  • .github/pull_request_template.md
  • .github/workflows/ci.yml
  • .github/workflows/release.yml
  • README.md
  • tests/e2e/visual.packaged.spec.ts
✅ Files skipped from review due to trivial changes (2)
  • .github/workflows/release.yml
  • README.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/pull_request_template.md

@chrisjcthomas chrisjcthomas merged commit 846b4ea into main Mar 22, 2026
5 checks passed
@chrisjcthomas chrisjcthomas deleted the codex/codex-review-setup branch March 22, 2026 00:55
@coderabbitai coderabbitai Bot mentioned this pull request Mar 22, 2026
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant