fix(#5188): re-trigger review via label after fix-agent push - #469
fix(#5188): re-trigger review via label after fix-agent push#469ggallen wants to merge 1 commit into
Conversation
PR Summary by QodoFix: retrigger review by relabeling after fix-agent push
AI Description
Diagram
High-Level Assessment
Files changed (6)
|
|
🤖 Finished Review · ✅ Success · Started 3:07 PM UTC · Completed 3:23 PM UTC |
Code Review by Qodo
Context used✅ Compliance rules (platform):
55 rules 1.
|
ReviewFindingsMedium
Low
Previous runReviewFindingsMedium
Low
Previous run (2)ReviewFindingsMedium
Low
Previous run (3)ReviewFindingsMedium
Low
Previous run (4)ReviewFindingsMedium
Low
Previous run (5)ReviewFindingsMedium
Low
Previous run (6)ReviewFindingsMedium
Low
Previous run (7)ReviewFindingsMedium
Low
Previous run (8)ReviewFindingsMedium
Low
Previous run (9)ReviewFindingsMedium
Low
Previous run (10)ReviewFindingsMedium
Low
Previous run (11)ReviewFindingsMedium
Low
Previous run (12)ReviewFindingsMedium
Low
Labels: PR modifies fix agent post-script and impacts review agent dispatch via label-based retrigger Previous run (13)ReviewFindingsMedium
Low
|
037fcf7 to
4550e46
Compare
|
🤖 Review · ❌ Terminated · Started 3:31 PM UTC · Ended 3:52 PM UTC |
|
cc @rh-hemartin @ifireball — this is the replacement PR for fullsend-ai/fullsend#5551, which I closed after your feedback there that the fix belongs in this repo (the actual production home for agent scripts) rather than fullsend-ai/fullsend's scaffold copy, which is being deleted by fullsend-ai/fullsend#5588. Same fix, re-implemented against the real |
|
🤖 Finished Review · ✅ Success · Started 3:31 PM UTC · Completed 3:52 PM UTC |
4550e46 to
9ee4cfd
Compare
|
🤖 Finished Review · ✅ Success · Started 4:04 PM UTC · Completed 4:36 PM UTC |
9ee4cfd to
c8a1b27
Compare
|
🤖 Finished Review · ✅ Success · Started 4:41 PM UTC · Completed 4:58 PM UTC |
waynesun09
left a comment
There was a problem hiding this comment.
Review-only pass — 4 findings verified against current HEAD (c8a1b27), deduplicated against existing bot/human comments on this thread. No approval or change request; a human reviewer should still weigh in given the protected-path requirement already flagged above.
Cleanup after verifying (fullsend-ai#469 review finding) that removing then re-adding a label does fire a fresh labeled webhook event. Signed-off-by: Greg Allen <gallen@redhat.com>
c8a1b27 to
7f9976c
Compare
|
🤖 Finished Review · ✅ Success · Started 7:32 PM UTC · Completed 7:46 PM UTC |
7f9976c to
dc9c09a
Compare
06abbfc to
a7ed33c
Compare
|
🤖 Finished Review · ✅ Success · Started 10:25 AM UTC · Completed 10:38 AM UTC |
waynesun09
left a comment
There was a problem hiding this comment.
Review-only pass — findings verified against current HEAD (a7ed33c), deduplicated against existing comments on this thread. No approval or change request.
Severity: HIGH — "Fixes" keyword will auto-close #5188 while two of its three named trigger paths remain broken
Verified via gh pr view 469 --json closingIssuesReferences: GitHub itself has parsed this PR as closing fullsend-ai/fullsend#5188 on merge (via the PR description's closing line and the fix(#5188): ... title). Fetching that issue directly confirms it is still OPEN, labeled priority/high, titled "Review agent skipped on bot-authored PRs due to collaborator permission check failing for GitHub App accounts," and its triage comment states the bug is is_event_actor_authorized(PR_USER_LOGIN) failing closed in reusable-dispatch.yml for pull_request_target opened/synchronize/ready_for_review events on ANY bot-authored PR — with the recommended fix being a routing-logic change to reusable-dispatch.yml/dispatch.yml in fullsend-ai/fullsend.
This PR lives entirely in fullsend-ai/agents and never touches that check. Its actual mechanism, retrigger_via_label, is called from post-fix.src.sh gated on NO_PUSH=false (post-fix.src.sh:373-374) — i.e. it fires only after the fix agent's own successful push. It does not address: (1) GitHub's native ready_for_review event (draft-to-ready transition) that the issue explicitly names — a different thing from the same-named ready-for-review label this PR toggles; (2) synchronize events from any push that isn't the fix agent's own post-script (e.g. a human or other automation pushing directly to a bot-owned PR branch), which is the scenario the issue's own originating evidence describes generically. No existing comment/thread on this PR raises this scope/auto-close concern.
Suggestion: Change "Fixes fullsend-ai/fullsend#5188" to "References fullsend-ai/fullsend#5188" (or similar non-closing phrasing) so the issue doesn't auto-close on merge, and post a comment on #5188 scoping exactly what this PR covers (only the fix-agent-push synchronize path) versus what remains open (the ready_for_review event, and synchronize events from any other pusher), so the remaining gaps stay tracked.
a7ed33c to
70202b0
Compare
|
🤖 Finished Review · ✅ Success · Started 3:34 PM UTC · Completed 3:51 PM UTC |
70202b0 to
de5681e
Compare
|
🤖 Finished Review · ✅ Success · Started 3:59 PM UTC · Completed 4:14 PM UTC |
pull_request_target.synchronize fires when the fix agent pushes, but its actor-identity authorization check is gated on the PR's original author, which for agent-authored PRs is the code agent's bot account regardless of who triggered this fix run. GitHub App bots have no collaborator role, so that check always fails closed and review is never re-dispatched after a fix-agent push (fullsend-ai/fullsend#5188). post-fix.src.sh now calls a new shared library, relabel-retrigger.lib.sh, which removes then re-adds the ready-for-review label after a successful push, forcing a fresh labeled webhook event. That path has no actor-authorization gate at all — label application itself already requires write access, so it needs no separate identity check — mirroring post-code.src.sh's existing handling of the PR-open case. GitHub does not fire a new labeled event when a label already present is simply re-added, hence the remove-then-add sequence. Verified live on a disposable PR in a personal fork: remove-then-add fires a second, genuinely distinct pull_request.labeled run; a plain re-add of an already-present label does not. Extracted as a library (retrigger_via_label) rather than inline script logic, per review feedback that this kind of dispatch-retrigger functionality should be reusable across agent post-scripts, not duplicated per-agent. The GitHub token is taken as an explicit argument and scoped to each gh invocation individually (GH_TOKEN="${token}" gh ...) rather than relying on the caller having exported GH_TOKEN into the shell environment at the right point in script execution — the library's parameter-passing design makes that class of ordering bug structurally impossible. retrigger_via_label checks whether the label was actually present before attempting removal, so a genuine --remove-label failure (while the label was present) escalates to a warning instead of being masked by the idempotent --add-label call silently no-op'ing as if the retrigger had succeeded. Added a bundled-script-has-relabel-retrigger presence check and a real NO_PUSH=false integration test (a genuine feature-branch commit pushed against a local bare repo standing in for GitHub) to post-fix-test.sh, so a transposed argument, wrong token, flipped NO_PUSH guard, or a deleted call site would fail CI instead of passing silently. This fix was originally attempted in fullsend-ai/fullsend#5551 against internal/scaffold/fullsend-repo/scripts/post-fix.sh, but that copy is being deleted from that repo (fullsend-ai/fullsend#5588) since agent scripts are now served from this repo via resolveAgentSource() — this repo's post-fix.sh is the actual production script, and it still had the original bug, unaffected by anything in #5551. Signed-off-by: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Greg Allen <gallen@redhat.com>
de5681e to
833d10d
Compare
|
🤖 Finished Review · ✅ Success · Started 4:22 PM UTC · Completed 4:37 PM UTC |
waynesun09
left a comment
There was a problem hiding this comment.
Review-only pass — 3 findings verified against current HEAD (833d10d), deduplicated against existing comments on this thread. No approval or change request.
| # self-heals. This is a narrow, accepted window, not engineered around here. | ||
| # --------------------------------------------------------------------------- | ||
| if [ "${NO_PUSH}" = "false" ]; then | ||
| retrigger_via_label "${REPO_FULL_NAME}" "${PR_NUMBER}" "ready-for-review" "${PUSH_TOKEN}" |
There was a problem hiding this comment.
Severity: HIGH
Unconditional label retrigger double-dispatches review on human-authored PRs.
retrigger_via_label is called unconditionally here on every fix-agent push, with no check of who originally opened the PR. Cross-referencing the live routing logic this interacts with (fullsend-ai/.fullsend/.github/workflows/dispatch.yml, whose "Determine stage" step is unchanged in this regard from fullsend-ai/fullsend's internal/scaffold/fullsend-repo/.github/workflows/dispatch.yml and .github/workflows/reusable-dispatch.yml): under pull_request_target, opened|synchronize|ready_for_review dispatches review via is_event_actor_authorized("${PR_USER_LOGIN}", triage) — keyed on the PR's original author, not who pushed — while labeled dispatches review whenever TRIGGERING_LABEL == "ready-for-review" with no actor-authorization check at all. has_repo_permission()'s role hierarchy means admin/maintain/write roles pass regardless of the requested minimum, so any human PR author with write+ permission — the exact prerequisite for this repo's own pull_request_review.changes_requested auto-fix-loop branch (which requires has_label("fullsend-fix") && has_repo_permission(PR_USER_LOGIN, write) for human-authored PRs) — automatically also passes the triage-level check synchronize uses.
That makes the double-dispatch not merely possible but guaranteed whenever that loop runs: synchronize already dispatches review successfully on the fix agent's push, and this unconditional call fires a second, independent review dispatch via the labeled path for the same push. The review workflow's concurrency group is fullsend-review-${source_repo}-${pr} with cancel-in-progress: true, and pre-review.sh has no head-SHA/already-reviewed dedup (it only skips merged/closed PRs) — so both runs actually start before one is cancelled, non-deterministically discarding whichever run's partial work loses.
TRIGGER_SOURCE can't gate this either: the pull_request_review.changes_requested branch sets TRIGGER_SOURCE="${REVIEW_USER_LOGIN}" (the review bot's own login) in both the bot-authored-PR and human-authored-PR cases, so is_bot_user("${TRIGGER_SOURCE}") is true either way and would get this exact case backwards. Neither the PR description's Scope section (which names two different, unrelated known gaps) nor docs/review.md's new paragraph (which describes the label workaround only in terms of agent-authored PRs) raise or accept this trade-off.
Suggestion: Gate retrigger_via_label on the PR's original author, not on TRIGGER_SOURCE: thread the PR opener's login into post-fix.src.sh as a new input from the calling workflow, and only call retrigger_via_label when that login would fail the same check synchronize uses (e.g. matches a bot-login pattern, or lacks qualifying collaborator permission) — mirroring dispatch.yml's own is_event_actor_authorized. Until that's threaded through, at minimum update the PR description and docs/review.md to explicitly document the double-dispatch trade-off for human-authored PRs undergoing the auto-fix loop, rather than presenting this fix as scoped purely to bot-authored PRs.
| # `ready-for-review`-labeled path used when the PR was first opened): | ||
| # source "${SCRIPT_DIR}/lib/relabel-retrigger.lib.sh" | ||
| # | ||
| # GitHub does not fire a new `labeled` event when a label already present |
There was a problem hiding this comment.
Severity: MEDIUM
Forced unlabeled event's side effects elsewhere are unverified, not just undocumented.
Every retrigger_via_label call removes ready-for-review then re-adds it, necessarily firing a genuine pull_request_target.unlabeled webhook in addition to the intended labeled one. fullsend-ai/agents/.github/workflows/fullsend.yaml itself subscribes to pull_request_target: types: [..., labeled, unlabeled] and forwards every such event, unfiltered by action, to fullsend-ai/.fullsend's dispatch.yml. That file's pull_request_target case statement has no unlabeled) branch today, so the event is currently a no-op there.
But this codebase also ships a documented, general-purpose CEL-trigger system for exactly this class of event: internal/harnessdispatch/input/ghaevent.go maps raw unlabeled actions to normalized transition.kind == "label_changed" with transition.label.action == "removed", and docs/normative/normalized-event/v1/README.md's own worked CEL example is event.transition.kind == "label_changed" && event.transition.label.name == "ready-to-code" && event.transition.label.action == "added". The symmetric trigger for action == "removed" on ready-for-review — exactly the event this library forces on every fix push — is already fully expressible with today's documented vocabulary, requiring only a per-repo config addition, no dispatch.yml code change.
So "no consumer reacts to this today" is a narrower and more fragile guarantee than the library implies: its doc comment documents live verification that the labeled re-fire works (lines 16-21) but says nothing about the unlabeled transition's consequences, and no test in this PR exercises real webhook/CEL-trigger delivery for it. Because this library is explicitly written to be reused by other agent post-scripts, this unverified assumption propagates to every future adopter.
Suggestion: Add a note to the library's doc comment (next to the existing "Verified live" note) stating plainly that the interim unlabeled event is a no-op only because no current dispatch.yml case or configured CEL trigger reacts to ready-for-review removal, with a pointer to that case statement and to the label_changed/removed CEL vocabulary, so that wiring up such a trigger anywhere becomes a deliberate, reviewed decision rather than a silent regression against this library's callers.
| fi | ||
| fi | ||
|
|
||
| local add_output |
There was a problem hiding this comment.
Severity: MEDIUM
Durable --add-label failure is silent beyond the Actions log, with no retry.
The only failure handling for the final --add-label call (lines 96-102) is _relabel_retrigger_warn, which emits only a gha_echo warning / echo ... >&2 — a GitHub Actions run-log annotation, never a PR comment — and there is no retry. This call can fail for reasons entirely independent of the concurrency scenario the adjacent "Concurrency note" comment (post-fix.src.sh:367-371) describes: a transient GitHub API error, secondary rate limit, or momentary incident hitting this call specifically, in an otherwise fully sequential, non-concurrent run, immediately after --remove-label succeeded.
In that case there is no "next successful fix push" to self-heal if this was the run's last iteration — the iteration-cap mechanism (post-fix.src.sh:456-473) applies needs-human at the point documented as where "the autonomous review→fix loop needs human direction," i.e. the loop is expected to stop. The PR is then left with ready-for-review genuinely absent (not merely "not re-triggered"), which per docs/code.md ("Also marks workflow state for humans and the retro agent") is itself a human-visible signal, with zero trace on the PR.
For comparison, this same script already has a more visible escalation path for a comparable failure: push failures use post_fail_to_pr, which (via report_post_failure_to_pr in post-failure-report.lib.sh) actually calls gh pr comment to post a real, PR-visible comment — retrigger_via_label's add-label failure has no equivalent. Note: pr-assignee.lib.sh's maybe_assign_pr/_pr_assignee_warn is not a counter-example of visible reporting — its failure path is the same log-only-warning pattern as this library's, not a PR comment.
Suggestion: Add a single bounded retry around the --add-label call specifically (this script already has this convention elsewhere, e.g. its push --force-with-lease retry), since an add-label failure is uniquely consequential compared to a remove-label failure. Independently, surface a durable add-label failure on the PR itself rather than only in the runner log — e.g. fold a note into process-fix-result.py's step-6 summary comment, which runs immediately after this call and already posts a PR-visible comment — so a missed re-dispatch is visible to whoever reads the PR, not only to someone checking the Actions log.
|
Closing in favor of a more direct upstream fix, not merging this workaround. While this PR was in review, @waynesun09 found that its Investigating that led to fullsend-ai/fullsend#5706, which was actually the correct root-cause fix all along: That makes this PR's |
|
🤖 Finished Retro · ✅ Success · Started 10:25 AM UTC · Completed 10:38 AM UTC |
gitlint's local pre-commit environment generates gitlint_rules/__pycache__/*.pyc, which has no repo-wide .gitignore entry and gets accidentally staged by `git add -A` — happened twice during review of fullsend-ai#469 (fix(#5188): re-trigger review via label after fix-agent push), split out here per review feedback that it was unrelated scope creep on that PR. Signed-off-by: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Greg Allen <gallen@redhat.com>
Retro: PR #469 — re-trigger review via label after fix-agent pushWhat happenedPR #469 was a human-authored workaround for fullsend-ai/fullsend#5188 (review agent skipped on bot-authored PRs because The PR went through 14 review runs across 13 commits over 4 days (July 27-31). During review, an external reviewer (waynesun09) found that the workaround fires unconditionally on every fix-agent push, causing double-dispatch on human-authored PRs where the existing What went well
Review quality deltaThe fullsend-ai-review[bot] found only LOW-severity findings across all 14 runs (GHA sanitization gap, stale section references, test integrity observations, misleading test name). All MEDIUM and HIGH findings came from other reviewers:
The review agent's correctness sub-agent produced zero findings on an 854-line diff touching shell scripts with complex error handling logic — a notable gap given the density of real bugs found by other reviewers. Existing issues receiving new evidenceAll improvement opportunities from this workflow are covered by existing open issues. New evidence from this retro:
|
Throwaway experiment for #469 review finding — will be reverted immediately after observing results. Signed-off-by: Greg Allen <gallen@redhat.com>
gitlint's local pre-commit environment generates gitlint_rules/__pycache__/*.pyc, which has no repo-wide .gitignore entry and gets accidentally staged by `git add -A` — happened twice during review of #469 (fix(#5188): re-trigger review via label after fix-agent push), split out here per review feedback that it was unrelated scope creep on that PR. Signed-off-by: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Greg Allen <gallen@redhat.com>
gitlint's local pre-commit environment generates gitlint_rules/__pycache__/*.pyc, which has no repo-wide .gitignore entry and gets accidentally staged by `git add -A` — happened twice during review of fullsend-ai#469 (fix(#5188): re-trigger review via label after fix-agent push), split out here per review feedback that it was unrelated scope creep on that PR. Signed-off-by: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Greg Allen <gallen@redhat.com>
Throwaway experiment for fullsend-ai#469 review finding — will be reverted immediately after observing results. Signed-off-by: Greg Allen <gallen@redhat.com>
Cleanup after verifying (fullsend-ai#469 review finding) that removing then re-adding a label does fire a fresh labeled webhook event. Signed-off-by: Greg Allen <gallen@redhat.com>
What this fixes
After the fix agent pushes a commit to a PR, the review agent is never re-dispatched (fullsend-ai/fullsend#5188).
pull_request_target.synchronizefires on that push, but the dispatch routing's actor-identity check is gated on the PR's original author, which for agent-authored PRs is always the code agent's bot account, regardless of who actually triggered this fix run. GitHub App bots have no collaborator role, so that check always fails closed.The fix
post-fix.src.shnow calls a new shared library,scripts/lib/relabel-retrigger.lib.sh(retrigger_via_label), which removes then re-adds theready-for-reviewlabel after a successful push, forcing a freshlabeledwebhook event. That path has no actor-authorization gate at all — applying a label already requires write access, so no separate identity check is needed — mirroringpost-code.src.sh's existing handling of the PR-open case. GitHub does not fire a newlabeledevent when a label already present is simply re-added, hence the remove-then-add sequence.Why a library, not inline logic
This was originally implemented inline in fullsend-ai/fullsend#5551. Review feedback there (from @ifireball) pointed out this kind of dispatch-retrigger functionality should be reusable across agent post-scripts, not duplicated per-agent — so it's extracted here as
retrigger_via_label, following this repo's existingscripts/lib/*.lib.shconvention (e.g.pr-assignee.lib.sh).The library also closes a real bug found during that PR's review: the GitHub token is taken as an explicit argument and scoped to each
ghinvocation individually (GH_TOKEN="${token}" gh ...), rather than relying on the caller having exportedGH_TOKENinto the shell environment at the right point in script execution. The original inline version broke exactly that way —GH_TOKENwas exported after these calls ran, so they silently authenticated with whatever token was ambient. Taking the token as a parameter makes that entire class of ordering bug structurally impossible here, rather than merely caught by a test.Why this repo, not fullsend-ai/fullsend
fullsend-ai/fullsend#5551 attempted this same fix against
internal/scaffold/fullsend-repo/scripts/post-fix.shin that repo. That copy is being deleted entirely by fullsend-ai/fullsend#5588, since agent scripts are now served from this repo at runtime viaresolveAgentSource(). This repo'spost-fix.shis the actual production script, and it still has the original #5188 bug — #5551 would have fixed a copy that's already effectively dead. Closed in favor of this PR.Scope
This PR only covers the fix-agent-push
synchronizepath described in fullsend-ai/fullsend#5188 — i.e.retrigger_via_labelfires frompost-fix.src.sh, gated on a successful push by the fix agent itself. It does not cover:ready_for_reviewevent (draft-to-ready transition), a different thing from the same-namedready-for-reviewlabel this PR toggles.synchronizeevents from any pusher other than the fix agent's own post-script (e.g. a human or other automation pushing directly to a bot-owned PR branch).Those remaining gaps stay open against fullsend-ai/fullsend#5188 — see the scoping comment there.
Test plan
bash scripts/relabel-retrigger-test.sh— new tests cover bothghcalls receiving the correct token, the remove/add sequence tolerating either call failing without hard-failing the script, and sanitization of injected::, percent-encoded%0A/%0D, and bare CR sequences in error output.GH_TOKEN=scoping from one call and confirmed the test fails; restored and confirmed it passes.bash scripts/post-fix-test.sh— no regressions.make check-bundle—post-fix.shis in sync withpost-fix.src.sh.make script-test— all relevant suites pass (pre-existinggitlint-forbidden-type-scope-test.shfailures in this environment are due togitlintnot being installed locally, unrelated to this change).References fullsend-ai/fullsend#5188 (partial — see Scope above). Supersedes fullsend-ai/fullsend#5551.