Skip to content

Add CI-level check for deprecated scaffold paths, covering fork PRs that the review agent skips #513

Description

@guyoron1

What happened

PR #3233 by external contributor csoceanu modified internal/scaffold/fullsend-repo/skills/docs-review/SKILL.md and internal/scaffold/fullsend-repo/skills/pr-review/sub-agents/docs-currency.md. The fullsend dispatch workflow ran at 08:25 UTC on Jul 7 but exited with 'No stage matched — skipping dispatch' because is_event_actor_authorized('csoceanu') returned false (fork PR: PR_HEAD_REPO=csoceanu/fullsend != PR_BASE_REPO=fullsend-ai/fullsend). Only Qodo code review ran, finding 2 valid code-quality issues but not the deprecated-location problem. The human reviewer rh-hemartin posted a redirect comment at 13:00 UTC (~5 hours later). The PR was closed on Jul 8. This is the 4th+ documented instance: PRs fullsend-ai#2927, fullsend-ai#2417, fullsend-ai#2979/fullsend-ai#2775, and fullsend-ai#3156 all had the same pattern.

What could go better

Issue #3476 proposes the review agent detect deprecated scaffold paths, but the review agent is gated behind is_event_actor_authorized() and never runs on fork PRs. Even if fullsend-ai#3476 is implemented, external contributors from forks will still get no automated guidance about the migration. This PR demonstrates the gap: 5 hours elapsed before a human manually redirected the contributor. A CI-level check that runs on all PRs (including forks) would provide universal coverage. Confidence is high — this is a recurring pattern with 4+ documented instances. The only uncertainty is shelf life: once the deprecated files are removed from the repo, the check becomes unnecessary.

Proposed change

Add a lightweight job or step to the existing CI workflow (which runs on all PRs including forks) that checks if any changed files match internal/scaffold/fullsend-repo/** and fails with a clear message directing contributors to fullsend-ai/agents. This is distinct from fullsend-ai#3476 (review-agent-only, authorized-contributor-only). Implementation: a step that runs git diff --name-only HEAD~1..HEAD (or uses the GH API to get changed files) and checks for the deprecated prefix. On match, post a comment or fail the check with: 'These files have moved to fullsend-ai/agents. Please open your PR there instead.' Allow exceptions for PRs that only modify scaffold migration tooling (e.g., version bumps). Related issues: fullsend-ai#3476 (review agent detection), fullsend-ai#3157 (AGENTS.md documentation).

Validation criteria

On a test PR touching internal/scaffold/fullsend-repo/ files from a fork, the CI check fails with a clear redirect message within the first CI run (< 5 minutes). The check does not block PRs that only modify scaffold migration tooling. Time-to-redirect drops from hours/days (human intervention) to minutes (automated CI). Verified by opening a test fork PR touching the deprecated path and confirming the check fires.


Generated by retro agent from fullsend-ai#3233

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions