Skip to content

feat(#831): pre-validate shim content before creating PRs - #859

Open
fullsend-ai-coder[bot] wants to merge 1 commit into
mainfrom
agent/831-shim-pre-validation
Open

feat(#831): pre-validate shim content before creating PRs#859
fullsend-ai-coder[bot] wants to merge 1 commit into
mainfrom
agent/831-shim-pre-validation

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown

Summary

Add pre-validation of rendered shim content against the TestShimLabeledEventFiltering alignment test contract in reconcile-repos.sh. This prevents the reconciliation bot from opening PRs that would fail CI — as happened in PR fullsend-ai#5772 where the config repo template lacked the ready- prefix guard and label-aware concurrency group.

Related Issue

Fixes #831

Changes

  • Add validate_shim_content() function to reconcile-repos.sh that decodes base64 shim content and checks for three required patterns: the github.event.action != 'labeled' guard, the startsWith(github.event.label.name, 'ready-') prefix filter, and the format('label-{0}' concurrency group structure
  • Add pre-loop validation that runs the check once on the base template before processing any repos, exiting with non-zero on failure (fail-fast)
  • Add per-repo validation in the stale-shim update path that validates header-merged content, skipping repos whose merged content would fail CI
  • Update the default test template to include the required alignment patterns
  • Add four new test cases covering broken template (missing ready- prefix), broken template (missing concurrency group), valid template proceeding to enrollment, and stale shim path validating header-merged content

Testing

  • All 10 script tests pass (bash internal/scaffold/fullsend-repo/scripts/reconcile-repos-test.sh)
  • Tests added for new validation logic (tests 5-8)
  • Secret scan passed

Checklist

  • PR title follows Conventional Commits (correct type, ! for breaking changes)
  • Commits are signed off (DCO) — human and human-directed agent sessions only

Closes #831

Post-script verification

  • Branch is not main/master (agent/831-shim-pre-validation)
  • Secret scan passed (gitleaks — c887fc47c59d8b87b5282ca44044d2933d30df41..HEAD)
  • PR body secret scan passed (gitleaks — no-git)
  • Pre-commit hooks passed (authoritative run on runner)
  • Tests ran inside sandbox

Add validate_shim_content() to reconcile-repos.sh that checks
rendered shim templates against the TestShimLabeledEventFiltering
alignment test contract before creating PRs. The function verifies
three required patterns: the labeled-event if: guard, the ready-
prefix filter, and the label-aware concurrency group structure.

Validation runs in two places:
1. Pre-loop: validates the base template once before processing
   any repos, failing fast with non-zero exit on broken templates.
2. Stale-shim path: validates header-merged content per-repo to
   catch regressions introduced by user header injection.

This prevents the reconciliation bot from opening PRs that would
fail CI, as happened in PR fullsend-ai#5772 where the template lacked the
ready- prefix guard and label-aware concurrency group.

Test coverage includes four new cases: broken template (missing
ready- prefix), broken template (missing concurrency group),
valid template passes and proceeds, and stale shim path validates
header-merged content correctly.

Note: pre-commit could not run in the sandbox due to network
restrictions (git fetch blocked). The post-script runs an
authoritative pre-commit check.

Closes #831
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 2, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 9:43 AM UTC · Completed 9:55 AM UTC
Commit: be3f8e8 · View workflow run →

@fullsend-ai-review

Copy link
Copy Markdown

Looks good to me

@fullsend-ai-review fullsend-ai-review Bot added the ready-for-merge All reviewers approved — ready to merge label Aug 2, 2026
@guyoron1

guyoron1 commented Aug 2, 2026

Copy link
Copy Markdown
Owner

/fs-fix

@fullsend-ai-coder

fullsend-ai-coder Bot commented Aug 2, 2026

Copy link
Copy Markdown
Author

🤖 Finished Fix · ✅ Success · Started 10:31 AM UTC · Completed 10:36 AM UTC
Commit: be3f8e8 · View workflow run →

@fullsend-ai-coder

Copy link
Copy Markdown
Author

🔧 Fix agent — iteration 1 (human-triggered)

No changes made. The PR was approved by the review bot with no findings, and the human /fs-fix trigger included no specific instruction. All tests pass and code is clean.

Disagreed (1):

  1. No actionable feedback to address: The review bot approved this PR with 'Looks good to me' and no findings. The human triggered /fs-fix without a specific instruction (instruction was 'none'). All 10 tests pass, the code follows repo shell scripting conventions, and the secret scan is clean. No changes are needed.

Tests: passed

Next steps:

  • /fs-review — request a re-review of the changes
  • /fs-fix <your instruction> — run another fix pass with specific guidance
  • Push commits directly — review re-runs automatically on push
    Updated by fullsend fix agent

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

This pull request has been automatically marked as stale because it has not had any activity in the last month. It will be closed in 2 weeks if no further activity occurs. Remove the stale label to reset the inactivity timer.

@github-actions github-actions Bot added the stale label Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-merge All reviewers approved — ready to merge stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reconcile-repos.sh: pre-validate shim content against alignment test expectations before creating PRs

1 participant