feat(ci): AI Action to review contributor pull requests

[Emiyaaaaa]

Summary

Adds a GitHub Action that automatically analyzes PR diffs and posts review suggestions when someone other than the repository owner opens or updates a PR.

What was added

• .github/workflows/pr-ai-review.yml — triggers on pull_request_target (opened / synchronize / reopened / ready_for_review)
• scripts/ci/ai_pr_review.py — fetches PR metadata + diff via GitHub API, calls an OpenAI-compatible LLM, and upserts a single PR comment

Behavior

• Skips: repository owner, all bots (Dependabot / Renovate / github-actions / Copilot / any *[bot] login or user.type == Bot), draft PRs
• Defense in depth: workflow if + script-level is_bot_author() both skip bots
• Reuses one comment per PR (marker <!-- agentflow-ai-pr-review -->) instead of spamming
• Security: does not check out or execute the PR head; only the base-branch script runs and the diff is read from the API

Required setup (after merge)

1. Add repository secret LLM_API_KEY (or OPENAI_API_KEY)
2. Optional repository variables:
   • LLM_BASE_URL (default https://api.openai.com/v1) — any OpenAI-compatible endpoint
   • LLM_MODEL (default gpt-4o-mini)

Without the secret, the workflow warns and exits without failing the PR.

Test plan

• Merge, then set LLM_API_KEY (and optional LLM_BASE_URL / LLM_MODEL)
• Open a non-draft PR from a non-owner account (or fork) and confirm an AI PR Review comment appears
• Push another commit to that PR and confirm the same comment is updated (not duplicated)
• Confirm owner / draft / Dependabot (and other bot) PRs are skipped

  

[Copilot]

🟡 Changes recommended

Paginate issue comments so existing marked reviews are updated instead of duplicated.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds an automated GitHub Action that analyzes eligible contributor PRs with an OpenAI-compatible LLM and updates a single review comment.

Changes:

• Adds the PR review workflow.
• Adds Python logic for diff analysis and comment management.
• Filters owner, bot, and draft PRs.

File summaries

File	Review summary
scripts/ci/ai_pr_review.py	Moderate issue: paginate issue comments to reliably find and update existing reviews.
.github/workflows/pr-ai-review.yml	Workflow configuration reviewed; no final comments.

Review details

Suppressed comments (4)

.github/workflows/pr-ai-review.yml:31

• These checks use github.actor, which is the user or bot that caused the event, not the pull request author. A human-authored PR synchronized by an automation bot (for example, a formatting or branch-update workflow) will therefore be skipped even though pull_request.user is human and is_bot_author() would allow it. Gate only on github.event.pull_request.user to match the documented author-based bot filtering.

      && github.actor != 'dependabot[bot]'
      && !endsWith(github.actor, '[bot]')

scripts/ci/ai_pr_review.py:183

• The final fallback treats any comment containing the marker as this action's comment, including a human-authored comment that can be created before the workflow runs. The update can then overwrite that comment (or fail if the token cannot edit it) instead of updating the action's own comment. Restrict the match to the authenticated workflow bot rather than returning the first marker-bearing comment.

        if AI_PR_REVIEW_MARKER in body:
            return int(comment["id"])

scripts/ci/ai_pr_review.py:50

• This new bot-classification gate has no automated tests, although the repository's Python CI runs a substantial pytest suite. Add unit tests covering user.type == Bot, [bot] logins, known bot aliases, and human authors so the defense-in-depth skip behavior cannot regress silently.

def is_bot_author(login: str, user_type: str | None = None) -> bool:
    name = (login or "").strip().lower()
    if not name:
        return False
    if (user_type or "").lower() == "bot":
        return True
    if name.endswith("[bot]"):
        return True
    return name in KNOWN_BOT_LOGINS

scripts/ci/ai_pr_review.py:170

• This request is hard-coded to the first 100 issue comments. If a PR already has more than 100 comments before the first review run, the marker can be on a later page; subsequent synchronize events will not find it and will create another review comment instead of updating the existing one. Follow the API pagination links (or otherwise search all pages) before creating a comment.

    url = f"{api_base}/repos/{owner}/{repo}/issues/{pr_number}/comments?per_page=100"

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.