Evidence for agents#193: PR experiments#42 post-review 422 on inline comment outside diff hunk

[fullsend-ai-retro]

What happened

PR fullsend-ai/experiments#42 review run 29511937629 completed its review successfully (APPROVE with a low finding about a stale .gitignore entry). The post-review script posted the review comment body, then attempted to submit the formal GitHub review with 1 inline comment attached. The inline comment referenced a line outside the PR diff hunk, causing GitHub to return 422 Unprocessable Entity. Log output: 1 finding(s) posted as file-level comment(s) (line outside diff hunk) followed by Error: submitting review: create pull request review on #42: github api: 422 Unprocessable Entity. Because the post-script exited with code 1, the ready-for-merge label was never applied. The author had to force-push (to fix a separate stale usage text finding) and manually trigger /fs-review to get a successful review in run 29512743867.

What could go better

This is the same root cause as agents#193: fullsend post-review does not validate inline comment line numbers against diff hunks before submitting. PR agents#195 addresses the contributing factor (apply labels even on submission failure) but the root cause in fullsend post-review itself still needs fixing. Confidence: high — the failure pattern matches exactly.

Proposed change

This is additional evidence for agents#193. The root cause fix should be in fullsend post-review: validate that each inline comment's line number falls within a diff hunk for that file before attaching it to the review submission. If a line is outside all hunks, either omit the inline comment or convert it to a file-level comment. PR agents#195 should also land to ensure labels are applied even when submission fails.

Validation criteria

After the fix lands, a review with findings only on lines outside diff hunks should: (1) submit the formal GitHub review without failing, (2) omit or convert out-of-hunk inline comments, and (3) apply the appropriate outcome label (ready-for-merge or needs-revision).

---

Generated by retro agent from fullsend-ai/experiments#42

[fullsend-ai-prioritize]

RICE Priority Score: 2.03

Score breakdown

Dimension	Score	Reasoning
Reach	1.5	This affects the automated review pipeline used across multiple projects. Two confirmed occurrences (agents#173 for issue #193, and experiments#42 for this issue) demonstrate the pattern recurs whenever a review finding lands on a line outside a diff hunk. Any project using the review agent is susceptible.
Impact	1.5	When triggered, the formal GitHub review submission fails with a 422, the outcome label (ready-for-merge) is not applied, and the author must manually force-push or re-trigger the review. This is a significant workflow disruption, though a manual workaround exists.
Confidence	0.9	Very high confidence. The issue is thoroughly documented with specific error logs, run IDs, and two independent occurrences confirming the same root cause. The proposed fix is well-scoped with clear validation criteria. The only minor gap is that this is a duplicate of #193 so the actual prioritization may be redundant.
Effort	1	The root cause fix requires validating each inline comment's line number against diff hunks before attaching it to the review submission in fullsend post-review. This is a well-scoped change — the existing code already filters by file, so extending to filter by hunk is incremental. PR #195 (closed) partially addressed the contributing factor of label application.

Formula: (1.5 x 1.5 x 0.9) / 1 = 2.03