ci(#388): fail roll-up when tests are skipped on pull_request_target

[rh-hemartin]

Summary

• On pull_request_target, when gate is skipped or auth is denied, all downstream jobs cascade to skipped. The roll-up only checked for failure/cancelled, so it reported SUCCESS -- letting PRs satisfy branch protection without any test running.
• Adds a check in the roll-up: on pull_request_target, if detect was skipped, fail. This covers both gate-skipped and auth-denied paths without affecting push/merge_group/workflow_dispatch.

Test plan

• Verify existing push/merge_group runs are unaffected (gate is legitimately skipped, detect runs)
• Verify pull_request_target with ok-to-test label still passes (gate runs, auth granted, detect runs)
• Verify pull_request_target without authorization now fails the roll-up instead of reporting SUCCESS

Closes #388

🤖 Generated with Claude Code

[qodo-code-review]

PR Summary by Qodo

CI: fail roll-up when pull_request_target tests are skipped

🐞 Bug fix ⚙️ Configuration changes 🕐 10-20 Minutes

AI Description

• Prevent branch protection from passing when pull_request_target test jobs are skipped.
• Fail the roll-up if detect is skipped on pull_request_target (gate skipped or auth denied).
• Keep push/merge_group/workflow_dispatch behavior unchanged.

Diagram

graph TD
  A["Workflow run"] --> B["Roll-up: Check results"] --> C{"on pull_request_target\nAND detect skipped?"}
  C -->|"yes"| D["Fail roll-up (exit 1)"] --> E["Branch protection blocked"]
  C -->|"no"| F["Evaluate gate/tests results"] --> G["Roll-up pass/fail"]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Fail roll-up on any skipped needs (generic rule)

• ➕ Catches future cases where other prerequisite jobs are skipped unexpectedly
• ➕ Simplifies logic to a single policy (no event-specific branching)
• ➖ Would likely break legitimate skip paths on push/merge_group/workflow_dispatch
• ➖ Higher risk of introducing noisy failures and CI friction

2. Move the authorization/skip check into detect (make detect always run)

• ➕ Keeps the roll-up purely aggregative
• ➕ Centralizes authorization semantics in the job that owns it
• ➖ May require reworking job conditionals/permissions to ensure detect truly runs on pull_request_target
• ➖ Still requires a required-check strategy to ensure merges are blocked when detect can’t run

3. Make detect (or gate) the required check instead of a roll-up

• ➕ Eliminates ambiguity where roll-up can hide skipped downstream jobs
• ➕ Uses GitHub’s native required check semantics per job
• ➖ May not fit existing required-check setup (single roll-up required check)
• ➖ Could increase required-check count and complicate branch protection configuration

Recommendation: The PR’s targeted roll-up guard is the best tradeoff: it fixes the branch-protection hole specifically for pull_request_target without changing skip semantics for other events. A generic “fail on any skipped” policy is more comprehensive but too risky for workflows that intentionally skip jobs; reworking detect/gate to always run is cleaner long-term but larger in scope.

Files changed (1) +5 / -0

Bug fix (1) +5 / -0

functional-tests.ymlFail roll-up when detect is skipped on pull_request_target +5/-0

Fail roll-up when detect is skipped on pull_request_target

• Adds EVENT_NAME to the roll-up step environment and introduces an early check that fails the roll-up if the workflow is running on pull_request_target and the detect job result is skipped. This prevents unauthorized or gate-skipped pull_request_target runs from reporting SUCCESS when no tests executed.

.github/workflows/functional-tests.yml

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 1:23 PM UTC · Completed 1:34 PM UTC
Commit: fe6ef26 · View workflow run →

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (1)

1. Protected workflow file modified 📜 Skill insight § Compliance

Description

This PR modifies .github/workflows/functional-tests.yml, which is a protected
governance/infrastructure path requiring explicit human review and must not be auto-approved. Ensure
the PR receives the required human approval (e.g., CODEOWNERS) before merge.

Code

.github/workflows/functional-tests.yml[R370-378]

+          EVENT_NAME: ${{ github.event_name }}
          GATE_RESULT: ${{ needs.gate.result }}
          DETECT_RESULT: ${{ needs.detect.result }}
          TESTS_RESULT: ${{ needs.functional-tests.result }}
        run: |
+          if [ "$EVENT_NAME" = "pull_request_target" ] && [ "$DETECT_RESULT" = "skipped" ]; then
+            echo "::error::Detect was skipped on pull_request_target — tests were not authorized to run"
+            exit 1
+          fi

Relevance

●●● Strong

Repo has treated .github workflow changes as governance-sensitive and asked for explicit
justification/human review before merging.

PR-#184
PR-#29

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The compliance rule requires raising a finding for any PR that modifies protected
governance/infrastructure paths (including .github/). The diff shows modifications in
.github/workflows/functional-tests.yml (added env var and a new failing condition), therefore this
must be treated as requiring human review and not auto-approved.

.github/workflows/functional-tests.yml[370-378]
Skill: pr-review

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Changes under `.github/` are protected governance/infrastructure modifications and must not be auto-approved.

## Issue Context
This PR updates a GitHub Actions workflow under `.github/workflows/`, which requires explicit human review/approval per policy.

## Fix Focus Areas
- .github/workflows/functional-tests.yml[370-378]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Label events fail rollup 🐞 Bug ≡ Correctness

Description

On pull_request_target, the roll-up now hard-fails when detect is skipped; because the workflow
triggers on any labeled event but gate is skipped unless the applied label is exactly ok-to-test,
adding any other label will skip gate/detect and fail functional-tests-complete (even for
already-authorized PRs). This can unexpectedly block merges when maintainers apply unrelated labels
after tests have passed.

Code

.github/workflows/functional-tests.yml[R375-378]

+          if [ "$EVENT_NAME" = "pull_request_target" ] && [ "$DETECT_RESULT" = "skipped" ]; then
+            echo "::error::Detect was skipped on pull_request_target — tests were not authorized to run"
+            exit 1
+          fi

Relevance

●● Moderate

Plausible edge case, but no close repo precedent on pull_request_target label-trigger behavior vs
ok-to-test gating.

PR-#89
PR-#184

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The workflow is configured to run on any pull_request_target labeled event, but gate is skipped
unless the applied label is ok-to-test; detect depends on gate authorization and becomes skipped,
and the new roll-up logic fails on detect==skipped for pull_request_target. The authorization script
would authorize any labeled event when ok-to-test is present, but gate never runs on those events
today.

.github/workflows/functional-tests.yml[27-28]
.github/workflows/functional-tests.yml[46-49]
.github/workflows/functional-tests.yml[76-81]
.github/workflows/functional-tests.yml[362-378]
.github/scripts/check-e2e-authorization.sh[104-110]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The workflow triggers on all `pull_request_target` label events, but the `gate` job is conditionally skipped unless the label being applied is `ok-to-test`. With the new roll-up rule (`pull_request_target` + `detect` skipped => fail), applying any other label causes `gate` and then `detect` to be skipped, which fails `functional-tests-complete` even when the PR is already authorized.

## Issue Context
The authorization script already supports authorizing *any* `labeled` event as long as the PR currently has `ok-to-test` (it checks the PR’s label set), but the `gate` job never runs for those events due to its `if:` filter.

## Fix Focus Areas
- .github/workflows/functional-tests.yml[46-49]
- .github/workflows/functional-tests.yml[27-28]

### Suggested change
Adjust the `gate.if` so it runs on `pull_request_target` labeled events whenever the PR has `ok-to-test` (not only when the *newly added* label is `ok-to-test`), e.g.:
- `github.event.action != 'labeled' || contains(github.event.pull_request.labels.*.name, 'ok-to-test')`

This prevents false failures from unrelated label changes while keeping the new roll-up failure for genuinely unauthorized PRs.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Gate failure masked ✓ Resolved 🐞 Bug ◔ Observability

Description

functional-tests-complete checks for detect==skipped before checking needs.gate.result, so if the
gate job fails/cancels (causing detect to be skipped), the roll-up exits with a misleading “not
authorized” message instead of reporting the gate failure. This makes diagnosing real gate errors
harder.

Code

.github/workflows/functional-tests.yml[R375-381]

+          if [ "$EVENT_NAME" = "pull_request_target" ] && [ "$DETECT_RESULT" = "skipped" ]; then
+            echo "::error::Detect was skipped on pull_request_target — tests were not authorized to run"
+            exit 1
+          fi
          if [ "$GATE_RESULT" = "failure" ] || [ "$GATE_RESULT" = "cancelled" ]; then
            echo "::error::Gate job ${GATE_RESULT}"
            exit 1

Relevance

●●● Strong

Improving failure messaging/diagnostics in workflows is routinely accepted; check gate result first
to avoid misleading authorization error.

PR-#184
PR-#90

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The roll-up’s first branch exits on detect==skipped, so the later gate failure/cancelled branch is
unreachable in that scenario; detect is skipped whenever the authorized output is not 'true', which
includes cases where gate did not succeed.

.github/workflows/functional-tests.yml[375-382]
.github/workflows/functional-tests.yml[76-81]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The roll-up step exits early on `DETECT_RESULT=skipped` for `pull_request_target`, before it checks `GATE_RESULT`. If `gate` fails/cancels and `detect` is skipped as a consequence, the workflow reports the wrong root cause.

## Issue Context
This doesn’t change pass/fail outcome (it still fails), but it obscures the actual failure mode and slows debugging.

## Fix Focus Areas
- .github/workflows/functional-tests.yml[375-385]

### Suggested change
Reorder the checks so gate failure/cancellation is handled before the `detect==skipped` authorization guard, or make the `detect==skipped` guard conditional on `GATE_RESULT == success` (and optionally add `AUTHORIZED=${{ needs.gate.outputs.authorized }}` to emit a precise message).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Context

✅ Compliance rules (platform): 55 rules

✅ Skills: 4 invoked

  code-review

  code-implementation

  pr-review

  docs-review

Tip of the day

💡 Did you know, you can route each action level your way: inline, summary, both, or drop

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[fullsend-ai-review]

Review

Findings

Medium

• [protected-path] .github/ — PR modifies files under the .github/ protected path: .github/scripts/check-rollup-result.sh, .github/scripts/check-rollup-result-test.sh, .github/workflows/functional-tests.yml. The change is authorized by issue ci(functional-tests): complete job reports SUCCESS when all tests are skipped #388 and the PR description explains the rationale (fix roll-up reporting SUCCESS when tests were skipped on pull_request_target). Human approval is always required for protected-path changes.

Low

• [test-coverage-gap] .github/scripts/check-rollup-result-test.sh — No test covers the scenario where detect succeeds but functional-tests is skipped due to an empty agent matrix (push/merge_group with no matching agents). In that case the rollup receives TESTS_RESULT=skipped on a non-PRT event and should pass. An explicit test case like assert_pass "push, tests skipped (no agents)" push skipped success skipped would document this intentional behavior.

• [scope-extension] .github/workflows/functional-tests.yml — The PR extracts inline bash logic to a separate script with checkout, test harness, and sparse-checkout optimization. Issue ci(functional-tests): complete job reports SUCCESS when all tests are skipped #388 describes a logic bug; the fix could have been applied inline. The extraction follows the repo's established CI scripting pattern and makes the new logic testable — reasonable engineering practice with no regression risk.

Previous run

Review

Findings

Medium

• [protected-path] .github/ — PR modifies files under the .github/ protected path: .github/scripts/check-rollup-result.sh, .github/scripts/check-rollup-result-test.sh, .github/workflows/functional-tests.yml. The change is authorized by issue ci(functional-tests): complete job reports SUCCESS when all tests are skipped #388 and the PR description explains the rationale (fix roll-up reporting SUCCESS when tests were skipped on pull_request_target). Human approval is always required for protected-path changes.

Previous run (2)

Review

Findings

Medium

• [protected-path] .github/ — PR modifies files under the .github/ protected path: .github/scripts/check-rollup-result.sh, .github/scripts/check-rollup-result-test.sh, .github/workflows/functional-tests.yml. The change is authorized by issue ci(functional-tests): complete job reports SUCCESS when all tests are skipped #388 and the PR description explains the rationale (fix roll-up reporting SUCCESS when tests were skipped on pull_request_target). Human approval is always required for protected-path changes.

Low

• [test-coverage-gap] .github/scripts/check-rollup-result-test.sh:48 — No test case exercises the detect=failure path (line 39-42 of check-rollup-result.sh) or tests=cancelled path (line 43-45). The test at line 65 covers detect=cancelled but not detect=failure. The test at line 49 covers tests=failure but not tests=cancelled. These are minor gaps since the bash conditionals use simple || with identical exit behavior for both failure and cancelled.
  Remediation: Add test cases such as: assert_fail "push, detect failure" push "" "" skipped failure skipped and assert_fail "push, tests cancelled" push "" "" skipped success cancelled.

Previous run (3)

Review

Findings

Medium

• [protected-path] .github/ — PR modifies files under the .github/ protected path: .github/scripts/check-rollup-result.sh, .github/scripts/check-rollup-result-test.sh, .github/workflows/functional-tests.yml. The change is authorized by issue ci(functional-tests): complete job reports SUCCESS when all tests are skipped #388 and the PR description explains the rationale (fix roll-up reporting SUCCESS when tests were skipped on pull_request_target). Human approval is always required for protected-path changes.

Low

• [error-message-accuracy] .github/scripts/check-rollup-result.sh:31 — Rule ordering causes misleading error diagnostics on pull_request_target when gate fails or is cancelled. Rule 2 (detect==skipped on PRT, line 31) fires before Rule 3 (gate failure/cancelled, line 36), so the error message says “tests were not authorized to run” when the actual cause is gate failure/cancellation. The exit code (1) is correct; only the diagnostic text is misleading.
  Remediation: Move the gate failure/cancelled check (lines 36–39) above the detect-skipped check (lines 31–34).

• [test-coverage-gap] .github/scripts/check-rollup-result-test.sh:63 — Test case “gate cancelled” does not exercise the gate-cancelled code path. With EVENT_NAME=pull_request_target and DETECT_RESULT=skipped, the script exits at Rule 2 (detect skipped on PRT) before reaching Rule 3 (gate cancelled). The test passes but for a different reason than the test name implies.
  Remediation: Either reorder checks in check-rollup-result.sh so gate checks precede detect checks, or rename the test and add a separate test that exercises Rule 3 directly.

Previous run (4)

Review

Findings

Medium

• [protected-path] .github/workflows/functional-tests.yml — PR modifies a file under the .github/ protected path. The change is authorized by issue ci(functional-tests): complete job reports SUCCESS when all tests are skipped #388 and the PR description explains the rationale (fix roll-up reporting SUCCESS when tests were skipped on pull_request_target). Human approval is always required for protected-path changes.

Low

• [error-message-format] .github/workflows/functional-tests.yml:376 — The new error message uses a static string rather than interpolating DETECT_RESULT, unlike the existing error messages which interpolate their respective variables (e.g., ${GATE_RESULT}, ${DETECT_RESULT}, ${TESTS_RESULT}). Since this branch only executes when DETECT_RESULT is exactly "skipped", interpolation would add redundant information — the message already says "was skipped." This is a minor stylistic inconsistency, not a functional gap.
  Remediation: Optionally interpolate for visual consistency: echo "::error::Detect was skipped on pull_request_target (${DETECT_RESULT}) — tests were not authorized to run".

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 6:40 AM UTC · Completed 6:55 AM UTC
Commit: d92338f · View workflow run →

[rh-hemartin]

/fs-fix rebase, resolve conflicts and address feedback. Don't sign the commits, bots shouyldn't do it.

[fullsend-ai-coder]

🤖 Finished Fix · ✅ Success · Started 10:04 AM UTC · Completed 10:08 AM UTC
Commit: d92338f · View workflow run →

[fullsend-ai-coder]

🔧 Fix agent — iteration 1 (human-triggered)

Rebased onto main (resolved Makefile conflict). Addressed 2 review findings by reordering checks in check-rollup-result.sh so gate failure/cancelled is detected before PRT detect-skipped, fixing both misleading error messages and test coverage gap. All 12 tests pass.

Fixed (3):

1. rebase onto main and resolve conflicts: Rebased branch onto origin/main, resolved Makefile conflict by keeping all three new test entries (process-fix-result-test.py, scrub-eval-results-test.sh, and check-rollup-result-test.sh)
2. error-message-accuracy: rule ordering causes misleading error diagnostics on PRT when gate fails (.github/scripts/check-rollup-result.sh): Moved gate failure/cancelled check above the PRT detect-skipped check so gate failures produce accurate diagnostics instead of the misleading 'tests were not authorized to run' message
3. test-coverage-gap: gate cancelled test does not exercise gate-cancelled code path (.github/scripts/check-rollup-result-test.sh): Addressed by the same reordering fix in check-rollup-result.sh — the gate cancelled test now correctly exercises the gate failure/cancelled check before the detect-skipped check fires

Disagreed (1):

1. protected-path: .github/ files require human approval: Informational finding, not actionable. The PR inherently modifies .github/ files as authorized by issue ci(functional-tests): complete job reports SUCCESS when all tests are skipped #388. Human approval is required per policy and the requires-manual-review label is already applied.

Tests: passed

Next steps:

• /fs-review — request a re-review of the changes
• /fs-fix <your instruction> — run another fix pass with specific guidance
• Push commits directly — review re-runs automatically on push
  _{Updated by fullsend fix agent}

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 10:10 AM UTC · Completed 10:21 AM UTC
Commit: 32a567b · View workflow run →

[waynesun09]

Review-only pass: 2 findings below (1 critical, 1 medium), both verified against head 32a567b and checked for duplicates against existing PR comments before posting.

[rh-hemartin]

/fs-fix

[fullsend-ai-coder]

🤖 Finished Fix · ❌ Failure · Started 7:07 AM UTC · Completed 7:14 AM UTC
Commit: 32a567b · View workflow run →

[fullsend-ai-coder]

⚠️ Post-fix script failed — Push rejected — workflows permission (exit code 1)

The fix agent completed, but the post-fix script failed before finishing.

Security boundary: the coder app intentionally lacks workflows write permission. Changes to .github/workflows/ must be made outside the agent (e.g., via a manual PR). Re-run the agent without workflow file changes, or apply those changes separately.

Workflow run: https://github.com/fullsend-ai/.fullsend/actions/runs/30792417204

Details:
To https://github.com/fullsend-ai/agents.git
! [remote rejected] agent/388-fix-skipped-tests-rollup -> agent/388-fix-skipped-tests-rollup (refusing to allow a GitHub App to create or update workflow .github/workflows/functional-tests.yml without workflows permission)
error: failed to push some refs to 'https://github.com/fullsend-ai/agents.git'
To https://github.com/fullsend-ai/agents.git
! [remote rejected] agent/388-fix-skipped-tests-rollup -> agent/388-fix-skipped-tests-rollup (refusing to allow a GitHub App to create or update workflow .github/workflows/functional-tests.yml without workflows permission)
error: failed to push some refs to 'https://github.com/fullsend-ai/agents.git'
Please check the workflow logs for full details and retry with /fs-fix if appropriate.

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 7:00 AM UTC · Completed 7:11 AM UTC
Commit: 8c285ec · View workflow run →

[waynesun09]

Review-only pass: 1 finding below (1 high), verified against head 8c285ec and checked for duplicates against existing PR comments/reviews before posting.

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 8:15 AM UTC · Completed 8:33 AM UTC
Commit: 831d310 · View workflow run →

[fullsend-ai-review]

See the review comment for full details.

[waynesun09]

Verified the extracted roll-up script at head 831d310. All findings from my earlier review passes are addressed:

• The labeled-event success bypass is gone — the script now fails whenever detect is skipped on pull_request_target, regardless of which label triggered the run (fail-closed, per the issue's recommended fix).
• Gate failure/cancelled is checked before the detect-skipped rule, so diagnostics report the true root cause (also covers the two open low-severity threads on ordering and the gate cancelled test path — the test now genuinely exercises the gate check).
• The roll-up checkout uses base-branch github.sha with persist-credentials: false, carries the guard comment, and the job now declares permissions: contents: read.
• Traced all trigger paths: push/merge_group/workflow_dispatch unaffected (detect-skipped rule is PRT-scoped); authorized PRT passes; unauthorized/label-skipped PRT now fails. Tests are wired into make script-test and pass on this head.

Residual known behavior: an unrelated label applied to an already-green PR will re-run and go red since gate skips — acceptable fail-safe trade-off; can be tuned later by having gate's if: check the PR's current label set.

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 9:02 AM UTC · Ended 9:17 AM UTC

Commit: 8b543ce · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 9:18 AM UTC · Completed 9:38 AM UTC

Commit: 0dee113 · View workflow run →

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 9:23 AM UTC · Completed 9:40 AM UTC

Commit: 0dee113 · View workflow run →

[fullsend-ai-review]

Review skipped — this PR is already merged.

The /fs-review command only reviews open pull requests.

_{Posted by fullsend post-review check}

[fullsend-ai-retro]

Retro: PR #400 — fail roll-up when tests are skipped on pull_request_target

Timeline

• Jul 23: Issue #388 filed. Code agent failed to push (GitHub App lacks workflows write permission). rh-hemartin manually opened PR #400 (4 files, +128/−17).
• Jul 23: Review agent posted only low-severity findings (error message formatting). qodo-code-review identified labeled-event bypass as Bug/Correctness.
• Jul 29: Human reviewer waynesun09 posted MEDIUM: decision logic untested, test plan omits labeled-event path.
• Jul 31: /fs-fix triggered → fix agent rebased, reordered checks (3 fixed, 1 disagreed). Review agent re-reviewed → only low findings again.
• Jul 31: waynesun09 found CRITICAL: labeled-event bypass reopens ci(functional-tests): complete job reports SUCCESS when all tests are skipped #388 hole (lines 23–26 unconditionally exit 0 for PRT+labeled+non-ok-to-test). Also MEDIUM: missing guard comment.
• Aug 3: /fs-fix triggered → failed (workflows permission, same root cause as initial code agent failure).
• Aug 5: waynesun09 found HIGH: missing contents: read permission on new checkout step under the workflow's zero-scope permissions: {}.
• Aug 7: rh-hemartin fixed. waynesun09 approved at 831d310. Review agent posted low scope-extension observation.
• Aug 17: Merged after 25 days.

Review quality gap

The review agent ran across 5 revisions and posted only low-severity findings (error message formatting, test coverage gaps, scope observation). The human reviewer caught:

• CRITICAL: Labeled-event bypass that reopened the original security hole (ci(functional-tests): complete job reports SUCCESS when all tests are skipped #388)
• HIGH: Missing contents: read permission on a checkout step
• MEDIUM (x2): Untested decision logic and missing guard comment

The PR was small (4 files, 128 additions) — well within review capacity. qodo-code-review also caught the labeled-event bypass on the initial commit, while the fullsend review agent did not. The review agent's correctness and security sub-agent instructions cover the analysis patterns needed to find these issues (runtime mechanism tracing, bidirectional permission audit), but the agent did not execute them with sufficient depth on this PR's workflow files.

Evidence for existing issues

This retro reinforces several open proposals rather than surfacing novel improvements:

• #394 (CI workflow correctness checks): The review agent lacked CI-specific analysis to trace pull_request_target.types: [labeled] through the gate job's if: condition cascade. This is the class of structural workflow correctness check that Add CI workflow correctness checks to review agent sub-agents #394 proposes adding.
• #689 (GHA platform semantic constraints): The labeled-event bypass involves understanding how GitHub Actions skips downstream jobs when an upstream job's if: evaluates false — a platform semantic the review agent doesn't check for.
• #46 (bidirectional permission analysis): The missing contents: read on a checkout step is the same class of under-privilege miss as Review agent missed vouch-check missing issues: write permission (caught by qodo) #46 (vouch-check missing issues: write). This is now the second confirmed occurrence of this pattern.
• #412 (severity elevation for PR-purpose contradiction): The review agent found test coverage gaps but rated them low. Since the PR's stated purpose was fixing a security bypass, test coverage gaps leaving the fix untested should have been elevated.
• #158 (pre-fix skip for workflow-only files): The Aug 3 fix agent run failed pushing workflow file changes. Pre-fix script should skip fix agent when review findings only affect workflow files #158's pre-flight check would have prevented this wasted run.
• #685 (prior-iteration findings on re-review): The review agent re-reviewed after human findings were posted but didn't verify their resolution. Review agent should explicitly resolve or persist prior-iteration findings on re-review #685 covers agent findings; extending it to verify human reviewer findings would add value.

Autonomy assessment

On CI workflow PRs with security implications, the review agent cannot yet be trusted for autonomous approval. The human reviewer was essential — without waynesun09's CRITICAL finding, the PR would have merged with the original security bypass intact. Implementing #394 and #689 (CI workflow analysis) and #46 (bidirectional permission analysis) would close the most impactful gaps.