What happened
On PR #2811, the review bot posted at least 6 separate status comments across multiple review runs: "Review cancelled" (00:58), "Finished Review, Success" (01:24), detailed findings from a different run (01:37), "Finished Review, Success" (11:24), "Finished Review, Success" (00:05), and "Review skipped" (00:26). The human author triggered /fs-review 3 times in the final 2 hours (23:51 push → 00:02 manual trigger → 00:23 manual trigger → 00:25 merge), suggesting they could not determine from the comment timeline whether the review had completed and what its current verdict was.
What could go better
The root cause is that each review dispatch creates a new independent status comment. When multiple dispatches run for the same PR (auto-triggered on push + manual /fs-review + concurrency-cancelled runs), the PR timeline fills with contradictory signals. A "Success" comment from one run is followed by findings from another run, or a "cancelled" message appears between two "Success" messages. The human cannot easily identify which status is current.
This is related to but distinct from existing issues: #2054 (summary comment should reflect findings — about comment content), #2746 (non-deterministic verdicts — about finding consistency), and #1014 (debounce rapid events — about reducing dispatch volume). The gap is in the presentation layer: even if dispatch volume and finding consistency improve, having one comment per dispatch will always create timeline noise when multiple runs occur.
Confidence: Medium-high. The 3 manual /fs-review triggers strongly suggest UX confusion, though the human may have had other reasons for re-triggering (e.g., wanting a fresh review after fixes).
Proposed change
Adopt a single mutable status comment pattern for the review bot, similar to how site-preview bots update a single comment. The review post-script should: (1) search for an existing review status comment on the PR (by a marker like <!-- fullsend-review-status -->), (2) update it in-place with the latest run's verdict, findings count, and timestamp, rather than creating a new comment. The comment should clearly distinguish workflow status ('completed', 'cancelled', 'in-progress') from review verdict ('approved', 'N findings posted', 'changes requested'). This change would be in the review post-script logic in fullsend-ai/fullsend.
Validation criteria
On the next 5 PRs with multiple review dispatches, the PR timeline should contain at most one review status comment (updated in-place) rather than N separate comments. Human /fs-review re-trigger rate after an initial review completes should decrease (baseline: this PR had 3 manual triggers for a simple docs change).
Generated by retro agent from #2811
What happened
On PR #2811, the review bot posted at least 6 separate status comments across multiple review runs: "Review cancelled" (00:58), "Finished Review, Success" (01:24), detailed findings from a different run (01:37), "Finished Review, Success" (11:24), "Finished Review, Success" (00:05), and "Review skipped" (00:26). The human author triggered
/fs-review3 times in the final 2 hours (23:51 push → 00:02 manual trigger → 00:23 manual trigger → 00:25 merge), suggesting they could not determine from the comment timeline whether the review had completed and what its current verdict was.What could go better
The root cause is that each review dispatch creates a new independent status comment. When multiple dispatches run for the same PR (auto-triggered on push + manual
/fs-review+ concurrency-cancelled runs), the PR timeline fills with contradictory signals. A "Success" comment from one run is followed by findings from another run, or a "cancelled" message appears between two "Success" messages. The human cannot easily identify which status is current.This is related to but distinct from existing issues: #2054 (summary comment should reflect findings — about comment content), #2746 (non-deterministic verdicts — about finding consistency), and #1014 (debounce rapid events — about reducing dispatch volume). The gap is in the presentation layer: even if dispatch volume and finding consistency improve, having one comment per dispatch will always create timeline noise when multiple runs occur.
Confidence: Medium-high. The 3 manual
/fs-reviewtriggers strongly suggest UX confusion, though the human may have had other reasons for re-triggering (e.g., wanting a fresh review after fixes).Proposed change
Adopt a single mutable status comment pattern for the review bot, similar to how site-preview bots update a single comment. The review post-script should: (1) search for an existing review status comment on the PR (by a marker like
<!-- fullsend-review-status -->), (2) update it in-place with the latest run's verdict, findings count, and timestamp, rather than creating a new comment. The comment should clearly distinguish workflow status ('completed', 'cancelled', 'in-progress') from review verdict ('approved', 'N findings posted', 'changes requested'). This change would be in the review post-script logic infullsend-ai/fullsend.Validation criteria
On the next 5 PRs with multiple review dispatches, the PR timeline should contain at most one review status comment (updated in-place) rather than N separate comments. Human
/fs-reviewre-trigger rate after an initial review completes should decrease (baseline: this PR had 3 manual triggers for a simple docs change).Generated by retro agent from #2811