Review agent should detect when a PR silently reverts recently-merged changes

[fullsend-ai-retro]

What happened

Two agent-created PRs touched overlapping files and merged within 2 hours:

1. PR #6261 merged at 2026-08-16 21:58 UTC, updating DefaultMintURL and DefaultPerRepoMintURL to https://mint.fullsend.sh in internal/cli/admin.go, internal/config/defaults.go, and pkg/e2etest/auth.go.
2. PR #6258 merged at 2026-08-16 23:47 UTC. Its large refactor (removing credential mode) touched the same 3 files and silently reverted all of #6261's changes through merge conflict resolution.
3. A human (rh-hemartin) discovered the regression ~1 day later and created PR #6290 to re-apply the reverted changes.

Neither the review agent, the merge queue, nor CI detected that PR #6258's merge undid recently-merged work. The review agent for PR #6258 reviewed the diff in isolation without checking whether the modified files had been recently changed by other merged PRs.

What could go better

The review agent could detect potential silent reverts by checking the git history of modified files for recently-merged changes. When a PR modifies a file that was changed by a PR merged in the last 48-72 hours, the review agent should compare the current PR's version of those lines against the recently-merged version and flag any cases where recent additions are being removed or overwritten.

This is distinct from existing issues:

• #4568 covers cross-referencing open sibling PRs, not recently-merged PRs.
• #4515 covers code agent stale branches at development time, not review-time detection.
• #2208 covers explicit revert PRs, not silent reverts through merge conflict resolution.
• #4880 covers PRs that explicitly mention "Replaces #N", not implicit reversions.

Confidence: High that this is a real gap. The scenario is reproducible whenever two agent PRs touch overlapping files and the second one's refactor overwrites the first's targeted changes. The 1-day detection delay shows no existing mechanism catches this.

Proposed change

Add a "recently-merged overlap" check to the review agent's correctness sub-agent (or as a dedicated pre-review analysis step in the review harness). The check should:

1. For each file in the PR's diff, query git log --since='72 hours ago' --merges --diff-filter=M -- <file> on the base branch to identify recently-merged PRs that modified the same file.
2. For each recently-merged PR found, compare the PR's proposed version of the file against the recently-merged version. Flag lines where the current PR removes or overwrites additions from the recent merge.
3. Surface these as a high-severity finding with context: "This PR modifies <file> which was changed by PR #N (merged X hours ago). The following changes from PR #N appear to be reverted: [summary]. If this is intentional, note why in the PR description."

This could be implemented as:

• A new check in the correctness sub-agent's prompt (in agents/review.md or the pr-review skill) instructing it to run git log on modified files before reviewing the diff.
• Or a pre-review script step in scripts/pre-review.sh that generates a "recently-merged overlap report" and passes it to the review agent as additional context.

Validation criteria

Simulate the scenario: create two PRs that modify overlapping files, merge the first, then submit the second for review. The review agent should flag the overlap and identify which lines from the recently-merged PR are being changed. Measure over the next 10 PRs that touch files modified by other PRs merged within 72 hours — the review agent should flag the overlap in at least 80% of cases.

---

Generated by retro agent from fullsend-ai/fullsend#6290

[fullsend-ai-prioritize]

RICE Priority Score: 2.13

Score breakdown

Dimension	Score	Reasoning
Reach	2	This affects all teams using the review agent for PRs, which is platform-wide. Any repo where multiple agent PRs touch overlapping files can silently revert recently-merged work. The incident involved the core fullsend repo and was discovered by a human a day later, suggesting this gap affects most active users across orgs.
Impact	2	Very high — when this occurs, merged changes are silently lost without any automated detection, causing regressions that may reach production. The 1-day detection delay in the reported incident demonstrates that no existing mechanism (review agent, merge queue, CI) catches this, making it a severe gap in the review pipeline's correctness guarantees.
Confidence	0.8	High confidence. The issue is exceptionally well-documented with specific PR numbers, timestamps, and a clear reproduction scenario. It carefully distinguishes itself from 4 related issues, proposes a concrete solution with validation criteria, and was generated from a real incident. The only minor uncertainty is around implementation approach (prompt vs. pre-review script).
Effort	1.5	Moderate complexity. Requires adding git history analysis (git log queries for recently-merged changes per file), diffing the PR's version against recently-merged versions to detect overwrites, and integrating this into the review agent's flow — either as a prompt addition to the correctness sub-agent or as a pre-review script step. Touches multiple components (review skill, possibly pre-review harness) and needs careful testing with the described simulation scenario.

Formula: (2 x 2 x 0.8) / 1.5 = 2.13