ci: migrate behaviour tests to single-org ephemeral repos

[ggallen]

Summary

Migrate behaviour tests from the current pool-of-12-orgs model to a single configurable org (fullsend-ai-test) with ephemeral UUID-named repos. This supersedes #6815, #6829, and PR #6820 — combining the single-org migration with pool machinery cleanup, vendoring removal, and driver simplification.

Motivation

The current infrastructure is:

• Complex: 12 halfsend-* orgs with distributed locking (e2e-lock repo), channel-based pool of fixed-name repos, singleflight-cached ensurer
• Slow: 50MB vendored binary pushed per repo via github setup --vendor
• Fragile: CreateRepo polls GetRepo (API-layer only) instead of GetRef (git-layer), causing 422 Tree SHA does not exist and 409 Git Repository is empty errors

Target Architecture

• One org via BEHAVIOUR_ORG env var (default fullsend-ai-test)
• Ephemeral repos named bt-{uuid[:8]}-{scenario-slug} (95-char limit to reserve 5 for -fork suffix)
• repos install --fullsend-ref as the only install path — no vendoring, no github setup
• No pool: no channel, no semaphore, no pre-allocation, no outstanding tracking
• No enrollment: preview mint deploys with PER_REPO_WIF_REPOS=*; repos install --mint-url writes the repo variable
• Simplified driver: CreateRepo(ctx, hint) / DeleteRepo(ctx, name) / Finalize(ctx)
• Simplified ensurer: single CreateRepo(ctx, org, hint) (name, error) method — generates name, creates repo, polls GetRef("heads/main") for git readiness, runs repos install, validates, waits for workflow
• Fork cleanup in scenario cleanup (steps/cleanup.go), not driver
• E2E_KEEP_REPOS=true to preserve repos for debugging

Changes

Infrastructure

1. Add BehaviourOrg() helper (BEHAVIOUR_ORG env var) to pkg/e2etest/testutil.go
2. Export tokenForOrg as TokenForOrg(cfg, org) in pkg/e2etest/auth.go
3. Add RunReposInstall to common/setup.go — temp manifest, --fullsend-ref, --mint-url
4. Add envFullsendRef() / prHeadSHAFromEvent() for ref resolution
5. Add ValidatePerRepoPostInstallRefPinned to validate.go

Ensurer (ensure.go)

6. Rewrite with single CreateRepo interface method
7. Replace awaitCreation (polls GetRepo) with awaitGitReady (polls GetRef("heads/main"))
8. Remove: ensured cache, singleflight.Group, InvalidateCache, resetRepo, awaitDeletion, ensureRepoExists

Driver (composed.go)

9. Remove channel pool, outstanding map, slot generation
10. CreateRepo(ctx, hint) delegates to ensurer; DeleteRepo(ctx, name) calls client.DeleteRepo directly
11. Finalize tears down mint only

Factory

12. Rename repopool_cfmint_previews.go → factory_cfmint.go; NewRepoPoolCFMintPreviews → NewCFMintFactory
13. Wildcard perRepoWIFRepos: "*" and workflowHostRepos: "*"
14. Remove buildRepoList, envPoolSize, DefaultPoolSize
15. Delete repopool_external_mint.go (dead code)

Terminology

16. Rename "enrolled" → "installed" in step definitions, error messages, and all 14 feature files
17. AllocateRepo/DeallocateRepo → CreateRepo/DeleteRepo on Driver interface
18. Remove LeasedRepoName from World — use RepoName

Cleanup

19. Fix resolveForkName and resolveHostRepoName for UUID names (remove hardcoded "test-repo" base)
20. Add fork deletion to steps/cleanup.go
21. Remove BEHAVIOUR_INSTALL_MODE / InstallMode / per-org tag handling
22. Add BEHAVIOUR_FULLSEND_REF to e2e.yml; remove BEHAVIOUR_INSTALL_MODE
23. Update docs/guides/dev/behaviour-testing.md

Not removed (still used by admin e2e)

• RunGitHubSetup in common/setup.go
• Pool infrastructure in pkg/e2etest/ (AcquireOrg, OrgPool, ReleaseLock)

Supersedes

• e2e: migrate behaviour tests from 12-org pool to single fullsend-ai-test org with repos install #6815 (single-org migration — conservative scope)
• CreateRepo should wait for git-layer readiness; delete pool recycling machinery #6829 (pool cleanup + CreateRepo readiness fix)
• PR ci(#6815): migrate behaviour tests to single-org ephemeral repos #6820 (implementation of e2e: migrate behaviour tests from 12-org pool to single fullsend-ai-test org with repos install #6815)

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 4:47 PM UTC · Completed 4:51 PM UTC

Commit: 74aebe0 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $0.81

[fullsend-ai-triage]

Triage summary

Severity: high — this resolves ongoing flakiness (#6701, #6702) and removes substantial infrastructure complexity.

Category: Infrastructure refactoring (behaviour test provisioning).

⚠ Workflow changes required: Item 22 modifies e2e.yml (add BEHAVIOUR_FULLSEND_REF, remove BEHAVIOUR_INSTALL_MODE). The code agent cannot modify workflow files under current permissions — that change will need a manual PR or maintainer action.

Related open issues

The following open issues describe symptoms of the current pool architecture that this migration would resolve or obsolete:

• behaviourtest: wait for permission propagation after pool repo recreate (dispatch sees no role, matrix empty) #6701 — permission propagation failures after pool repo recreate
• behaviourtest: pool-org installation token exhausts GitHub API budget ~10 min into the suite #6702 — pool-org installation token exhausts API budget
• test(behaviour): delete leased pool repos after scenarios and recreate on acquire #6120 — delete/recreate leased pool repos

Consider closing those as superseded once this lands.

Proposed test verification

1. Repos are created as bt-{uuid[:8]}-{scenario} in the configured org
2. repos install --fullsend-ref is the sole install path (no vendoring)
3. CreateRepo polls GetRef("heads/main") — no 422/409 races
4. Cleanup deletes repos (and forks) unless E2E_KEEP_REPOS=true
5. Fork resolution works with UUID-based names (no hardcoded "test-repo")
6. No pool channels, semaphores, or distributed locks remain in behaviour paths

---

Labels: Infrastructure refactoring of e2e test provisioning — adding type/chore and component/ci for the workflow file changes; existing component/e2e and priority/high are correct.

---

Next steps:

• /fs-code — agent creates a PR to implement this issue
• /fs-code <your instruction> — agent implements with your specific guidance

[ggallen]

/fs-code

[fullsend-ai-coder]

🤖 Finished Code · ✅ Success · Started 5:34 PM UTC · Completed 5:55 PM UTC

Commit: 74aebe0 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $10.81