Skip to content

Slice 10 — Reviewer Agent posts a review on every newly opened PR #10

@safayavatsal

Description

@safayavatsal

What to build

The Reviewer Agent — every PR, human-authored or Agent-authored, gets an automated review by default.

This is HITL because review tone, format, and what counts as "worth flagging" are product-defining. A Reviewer Agent that's noisy or annoying actively damages the wedge.

  • agent-orchestrator consumes pr.opened events. For each event, the Reviewer Agent runs against the PR diff and posts comments back to platform-api.
  • ReviewerAgent is implemented as a deep module: (diff, context) → []ReviewComment. Pure over its inputs once a ModelClient is injected.
  • Comments support inline anchoring ({file, line, side}) when the model can identify a precise location, otherwise a top-level comment.
  • All Reviewer Agent comments are clearly attributed (author = forge-reviewer, special UI badge).
  • Review focus at MVP: correctness regressions, obvious security issues (e.g., shell injection, leaking secrets), and ambiguity that suggests the change is incomplete. Not style nitpicks.

Decisions to nail in this slice

  • Review prompt strategy and what the Reviewer is told not to comment on.
  • Comment volume cap (e.g., max N comments per PR) to avoid spam.
  • Behavior when the diff is too large to fit in context.

Acceptance criteria

  • Opening a PR (in slices 5 or 7's flow) triggers a review within bounded time.
  • The review either posts inline comments anchored to specific lines or a top-level comment with a summary.
  • All Reviewer Agent comments are visually distinct from human comments in the web UI.
  • On a trivial no-op PR, the review correctly says "looks fine" rather than inventing issues.
  • On a PR that introduces a deliberate bug from the eval set, the review flags the bug.
  • ReviewerAgent has tests against fixed diffs with a recorded-response ModelClient, asserting comment shape (not exact wording).

Blocked by

Metadata

Metadata

Assignees

No one assigned

    Labels

    hitlHuman-in-the-loop — needs design call before implementationtracer-bulletVertical slice through all integration layers

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions