fix(#6105): retry GetFileContent on 404 in post-install validation

[fullsend-ai-coder]

Summary

• Add retry-on-404 logic to ValidatePerRepoPostInstall in pkg/behaviourtest/drivers/install/validate.go to tolerate GitHub API read-after-write eventual consistency
• New getFileWithRetry helper retries GetFileContent up to 5 times with 2-second backoff when the error is a 404 (forge.IsNotFound); non-404 errors fail immediately
• Add unit tests covering retry success, retry exhaustion, non-404 immediate failure, and context cancellation; add integration-level tests for ValidatePerRepoPostInstall with transient 404s

Context

TestBehaviourSuite intermittently fails during post-install validation with a 404 reading back .github/workflows/fullsend.yaml immediately after fullsend github setup commits it. This caused merge queue ejections (e.g., PR #5736). The retry budget (5 attempts, 2s apart = ~8s max) matches the existing waitForFileAccessible pattern in url_dispatch.go.

Test plan

• All new getFileWithRetry unit tests pass (transient 404, exhausted retries, non-404 immediate failure, context cancellation)
• ValidatePerRepoPostInstall integration tests pass (retry success, clear error on exhaustion)
• All existing tests in pkg/behaviourtest/drivers/install/... pass (42 tests, 0.078s)
• go vet passes
• CI e2e behaviour suite validates the fix under real GitHub API conditions

---

Closes #6105

Post-script verification

• Branch is not main/master (agent/6105-validate-retry-on-404)
• Secret scan passed (gitleaks — d62fc7141712252d51853fa6a9be5af6b929251b..HEAD)
• PR body secret scan passed (gitleaks — no-git)

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 3:23 PM UTC · Completed 3:34 PM UTC

Commit: b27be69 · View workflow run →

[codecov]

Codecov Report

❌ Patch coverage is 89.47368% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/behaviourtest/drivers/install/validate.go	89.47%	0 Missing and 2 partials ⚠️

📢 Thoughts on this report? Let us know!

[fullsend-ai-review]

Looks good to me

---

Labels: PR fixes e2e test flakiness in the install driver's post-install validation

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 4:03 PM UTC · Completed 4:19 PM UTC

Commit: b27be69 · View workflow run →

[fullsend-ai-retro]

Retro: PR #6106 — retry GetFileContent on 404 in post-install validation

Workflow quality: Excellent. This was a clean end-to-end autonomous workflow with zero rework, first-attempt code success, and full agent–human alignment on review.

Timeline

Time (Aug 11)	Event
15:01	ralphbean filed #6105 — post-install validation races GitHub API eventual consistency
15:02–15:06	Triage agent (run 31504842381) — labeled type/bug, component/e2e, component/install, applied ready-to-code
15:06–15:22	Code agent (run 31505271442) — produced single-commit fix, opened PR #6106
15:22–15:34	Review agent (run 31506647181) — approved after correctness/intent-coherence/style sub-agents found no issues
15:26	Codecov: 89.47% patch coverage (above 80% threshold)
21:08	ralphbean approved (human review), added to merge queue
21:36	Merge queue rejection #1 — GCP WIF quota exhaustion in E2E tests
Aug 12 21:07	Re-queued; rejection #2 at 21:13 — same WIF quota issue
Aug 13 15:49	Re-queued; merged at 16:00

What went well

• First-attempt success. Code agent produced correct, well-tested code in ~15 minutes. The getFileWithRetry helper correctly matches the existing waitForFileAccessible pattern. No fix agent runs needed.
• Proactive test discipline. The agent noticed retry delays slowed 11 existing tests from 0s to ~8s each and added speedUpValidateRetries(t) to all affected tests, keeping the package at 0.078s.
• Thorough review. The correctness sub-agent made 20 tool calls verifying retry bounds, context cancellation, cleanup ordering, and test double embedding. The challenger pass validated both info-level findings.
• Zero agent–human delta. Both the review agent and ralphbean approved with no substantive findings. Independent code analysis confirms no missed issues.

Merge queue delay (not an agent workflow issue)

The 48-hour gap between human approval and merge was caused by two merge queue ejections from GCP WIF quota exhaustion (HTTP 429) in E2E test setup, plus a GitHub API timeout. The PR's own code was not the cause. Re-queuing happened manually ~once per day.

Evidence for existing open issues

• E2E tests lack GCP resource quota protection under merge queue retry pressure #6129 (E2E tests lack GCP resource quota protection under merge queue retry pressure): Both merge queue rejections were caused by WIF quota exhaustion — this PR is direct evidence of the impact.
• Reuse WIF providers across merge-queue attempts instead of recreating on every run #6179 (Reuse WIF providers across merge-queue attempts): The second rejection hit the same quota wall, suggesting WIF provider recreation on each queue attempt amplifies the problem.
• Post diagnostic comment and alert when a PR is ejected from the merge queue #2837 (Post diagnostic comment when PR ejected from merge queue): Neither ejection produced a diagnostic comment — ralphbean had to discover the failures and manually re-queue.
• forge: github/gitlab client retry backoff sleeps for real in unit tests #6056, layers: EnrollmentLayer poll/retry loops sleep for real in unit tests #6057, sandbox: CreateWithRetry sleeps for real between retry attempts in unit tests #6060 (retry loops sleep for real in unit tests): PR fix(#6105): retry GetFileContent on 404 in post-install validation #6106 adds another instance of the package-level-var-override pattern these issues aim to standardize.
• Review agent: flag inconsistent retry/backoff strategies within a file #3441 (flag inconsistent retry/backoff strategies): Another retry call site (5 attempts, 2s delay) that would benefit from a centralized retry utility.

Autonomy readiness

This PR demonstrates zero agent–human review delta on a well-scoped test-infrastructure bug fix with comprehensive tests and all CI green. The class of change — single-issue driver fix following established patterns, with 89% patch coverage — is a candidate for reduced human review requirements. See #3026 for the broader tracking effort.

No proposals filed

All improvement areas identified are already tracked by existing open issues. No new proposals are warranted.