Review-to-fix dispatch chain fails silently after bot-originated reviews

[fullsend-ai-retro]

What happened

On PR #6416, the review-to-fix auto-dispatch chain failed to trigger after bot-originated reviews, requiring human intervention to continue the pipeline.

Three data points are consistent:

1. Review docs: Add agent-compatible code problem document #1 (run 32390818302, triggered by code agent opening the PR) found critical findings at 16:31 UTC. No fix agent was auto-dispatched. The human had to manually trigger /fs-fix at 16:43 UTC.
2. Review Add problem areas: Tekton pipeline review, migration path, multi-tenancy #2 (run 32394890453, triggered by fix agent pushing code) found the same critical findings at 17:15 UTC. No fix agent was auto-dispatched. The PR sat untouched for 14 hours.
3. Review docs: Add codebase context problem document and trim CLAUDE.md #3 (run 32458466095, triggered by a human push the next morning) found the same findings at 07:40 UTC. Fix agent (run 32459660822) was auto-dispatched within minutes.

Pattern: reviews triggered by bot-originated events (code agent PR open, fix agent push) never produced a fix dispatch. The review triggered by a human push did.

What could go better

The review-to-fix auto-dispatch chain should work end-to-end regardless of whether the originating push was from a bot or human. The current behavior forces human intervention to restart the pipeline, adding latency (14 hours in this case) and defeating the purpose of automated fix iterations.

Confidence: moderate. The 3-for-3 pattern (bot reviews never dispatch, human review does) is clear, but I cannot determine the exact root cause from the available data. Possible explanations: (1) GitHub webhook suppression for events triggered by bot-originated workflows -- though GitHub Apps should typically be exempt from this; (2) the review agent submitted COMMENTED instead of CHANGES_REQUESTED for reviews 1 and 2, meaning no fix dispatch was expected -- but both reviews found critical issues, making COMMENTED unlikely; (3) concurrency groups or other workflow conditions prevented the fix from starting. The original review verdicts are masked by DISMISSED state (superseded by later reviews), so I cannot verify which explanation is correct.

Related but distinct from #5937 (debouncing redundant fix processing), #6050 (detecting missing dispatches from dropped webhooks), and #5666 (synchronize events not routed to review). None of those identify this specific failure mode: reviews after bot pushes silently fail to trigger fix dispatches.

Proposed change

First, determine the root cause by checking one of: (a) the webhook delivery log for the review events on PR #6416 (Settings > Webhooks > Recent Deliveries), or (b) the original review verdicts for runs 32390818302 and 32394890453 in the workflow logs (look for the review submission API call).

If the root cause is webhook suppression: add an explicit fix-agent dispatch step to the review post-script (scripts/post-review.sh in fullsend-ai/agents). After the review agent submits a CHANGES_REQUESTED verdict, the post-script should check whether the current workflow was triggered by a bot event. If so, dispatch the fix workflow via gh workflow run or repository_dispatch (these event types are not subject to webhook suppression). The fix workflow in fullsend-ai/fullsend would need to accept repository_dispatch as an additional trigger.

If the root cause is the review agent submitting COMMENTED instead of CHANGES_REQUESTED for critical findings: fix the review agent's verdict logic to always submit CHANGES_REQUESTED when critical or high-severity findings are present.

Either fix should respect the iteration cap from #902.

Validation criteria

On the next 5 PRs where the fix agent pushes code and the subsequent review finds critical or high-severity findings: the fix agent should be auto-dispatched within 5 minutes of the review verdict, without requiring a human push or /fs-fix command. Track the time between review verdict and fix agent start for bot-originated vs human-originated review chains -- the median should be comparable (currently ~2-3 minutes for human-originated chains).

---

Generated by retro agent from #6416

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 10:15 AM UTC · Completed 10:20 AM UTC

Commit: a313d79 · View workflow run →

[fullsend-ai-triage]

Triage summary

Severity: high — this silently breaks the automated review-to-fix loop for all bot-originated reviews, forcing human intervention and adding hours of latency.

Root cause investigation required. The 3-for-3 pattern is compelling but the root cause is masked because the original review verdicts on runs 32390818302 and 32394890453 are now DISMISSED. Two avenues to determine root cause:

• Webhook delivery logs (Settings → Webhooks → Recent Deliveries) for the pull_request_review events on PR fix(#6412): limit sandbox name to 19 characters for OpenShell #6416
• Workflow logs for those runs — look for the review submission API call to confirm CHANGES_REQUESTED vs COMMENTED

⚠ Workflow file changes likely required. The proposed fix (adding repository_dispatch as a trigger to the fix workflow) involves modifying files under .github/workflows/. The code agent cannot modify workflow files under current permissions — manual intervention (human PR or maintainer action) will be required.

Cross-repo coordination: If the root cause is webhook suppression, the fix spans two repos — fullsend-ai/agents (post-review script) and fullsend-ai/fullsend (fix workflow trigger). Coordinate changes accordingly.

Related issues (distinct, not duplicates):

• Auto-dispatch fix agent on critical review findings for bot-authored PRs #2596 — Auto-dispatch fix for bot-authored PRs (feature request, different failure mode)
• Detect PRs with missing review dispatch caused by silently dropped webhooks #6050 — Detecting missing review dispatch from dropped webhooks (different stage of pipeline)
• Define fix agent auto-triggering strategy with debounce to prevent redundant processing #5937 — Fix agent debounce strategy (complementary)
• Add circuit breaker to review-fix feedback loop #902 — Circuit breaker for review-fix loops (fix must respect this)

# Proposed test: verify fix dispatch after bot-originated review
def test_fix_dispatch_after_bot_review():
    # Given: a review triggered by a bot push finds critical issues
    review_trigger = "bot_push"  # code agent or fix agent
    review_verdict = "CHANGES_REQUESTED"
    findings_severity = "critical"

    # When: the review completes and submits its verdict
    dispatch_result = simulate_post_review_dispatch(
        trigger_source=review_trigger,
        verdict=review_verdict,
        max_severity=findings_severity
    )

    # Then: fix agent should be dispatched
    assert dispatch_result.fix_agent_dispatched is True
    assert dispatch_result.dispatch_latency_seconds < 300  # within 5 min

    # And: iteration cap is respected
    assert dispatch_result.iteration_count <= MAX_FIX_ITERATIONS

---

Labels: This is a bug in the review-to-fix dispatch chain involving both the review and fix agents, scoped to the dispatch/workflow-trigger component.

---

Next steps:

• /fs-code — agent creates a PR to implement this issue
• /fs-code <your instruction> — agent implements with your specific guidance

[fullsend-ai-prioritize]

RICE Priority Score: 1.4

Score breakdown

Dimension	Score	Reasoning
Reach	2	This breaks the automated review-to-fix dispatch chain for all bot-originated reviews, which is a core part of the platform's automated pipeline. Every PR that goes through automated code agent or fix agent push cycles is affected — this likely covers most active users across orgs who rely on the automated fix iteration loop.
Impact	1.5	The failure forces human intervention to restart the pipeline, adding significant latency (14 hours observed in the reported case) and defeating the purpose of automated fix iterations. A workaround exists (manually triggering /fs-fix), which prevents this from being fully blocking, but it's a significant pain point that degrades a core automated workflow.
Confidence	0.7	The issue is exceptionally well-documented with a clear 3-for-3 pattern (bot-originated reviews never dispatch, human-originated review does) and the triage agent confirmed high severity. However, the root cause is still unknown — it could be GitHub webhook suppression, incorrect review verdict submission (COMMENTED vs CHANGES_REQUESTED), or concurrency group conflicts. This root cause ambiguity lowers confidence slightly.
Effort	1.5	Requires root cause investigation across webhook delivery logs and workflow run logs, then a fix that likely spans two repositories (fullsend-ai/agents post-review script and fullsend-ai/fullsend workflow triggers). Workflow file changes require special permissions (noted by triage agent as needing manual/maintainer action). Cross-repo coordination, the investigation phase, and validation across 5 PRs make this a moderate-complexity effort.

Formula: (2 x 1.5 x 0.7) / 1.5 = 1.4