perf(#2351): batch path-existence checks via Git Trees API - #2360
Conversation
Add forge.Client.ListRepositoryFiles to retrieve all file paths in a repository's default branch with a single Git Trees API call (refs → commit → tree?recursive=1). This replaces the O(N) GetFileContent pattern used by ComparePathPresence, reducing 100+ sequential API calls to 3 fixed calls regardless of path count. Changes: - forge.Client: add ListRepositoryFiles(ctx, owner, repo) - github.LiveClient: implement using Git Trees API (reuses the same refs/commits/trees pattern as CommitFiles) - forge.FakeClient: implement using FileContents map keys - scaffold.ComparePathPresence: new batch implementation that calls ListRepositoryFiles once and checks membership locally - Tests: 6 ComparePathPresence tests including a guard that GetFileContent is never called; error injection and thread safety coverage for the new forge method PR #1954 introduces a naive ComparePathPresence in vendormanifest.go that loops GetFileContent per path. When that PR merges, its version should be replaced with this batch implementation. Closes #2351
E2E tests are runningAuthorization passed for this commit. See the E2E Tests workflow for results. |
Site previewPreview: https://bab75538-site.fullsend-ai.workers.dev Commit: |
|
🤖 Finished Review · ✅ Success · Started 7:30 PM UTC · Completed 7:41 PM UTC |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
ReviewFindingsMedium
Low
Info
|
|
/fs-fix resolve merge conflicts, address review findings, and make codecov happy please |
|
🤖 Finished Fix · ✅ Success · Started 10:43 AM UTC · Completed 11:05 AM UTC |
- Fix retryOnTransient -> retryOnRepoRace in ListRepositoryFiles - Add ErrTreeTruncated sentinel error for truncated tree responses - Update ListRepositoryFiles interface docs with truncation behavior - Remove GitHub-specific reference from pathpresence.go comment - Move ComparePathPresence from vendormanifest.go to pathpresence.go using batch ListRepositoryFiles instead of per-path GetFileContent - Add httptest-based tests for ListRepositoryFiles (success, truncated, not-found, empty-repo) to improve codecov patch coverage - Add TestComparePathPresence_TreeTruncated for sentinel error propagation - Remove stale ComparePathPresence tests from vendormanifest_test.go Addresses review feedback on #2360
|
/fs-fix resolve merge conflicts, and make codecov happy please |
|
🤖 Finished Fix · ✅ Success · Started 11:07 AM UTC · Completed 11:33 AM UTC |
Merge origin/main into branch, resolving conflicts in: - internal/forge/forge.go: keep ErrTreeTruncated (needed by ListRepositoryFiles) - internal/forge/fake.go: keep FakeClient.ListRepositoryFiles - internal/forge/github/github_test.go: keep ListRepositoryFiles tests - internal/scaffold/vendormanifest.go: discard main's naive ComparePathPresence (batch version lives in pathpresence.go) - internal/scaffold/vendormanifest_test.go: discard main's naive tests (batch tests live in pathpresence_test.go) Add tests to improve codecov patch coverage: - TestIsTreeTruncated: cover the new error sentinel helper - TestFakeClient_ListRepositoryFiles: cover FakeClient implementation - TestFakeClient_ListRepositoryFiles_ConcurrentSafe: verify thread safety - TestListRepositoryFiles_RefError: cover ref lookup failure path - TestListRepositoryFiles_CommitError: cover commit fetch failure path - TestListRepositoryFiles_TreeFetchError: cover tree fetch failure path Addresses review feedback on #2360
|
🤖 Finished Retro · ✅ Success · Started 12:28 PM UTC · Completed 12:38 PM UTC |
|
PR #2360 batched path-existence checks via the Git Trees API, replacing O(N) sequential GetFileContent calls with a single ListRepositoryFiles call. The code agent created the initial implementation on June 16, the review agent found valid findings (1 medium, 5 low, 4 info) within 15 minutes, and the PR then sat idle for 21 days until a human engaged on July 7. The human triggered two /fs-fix iterations: the first addressed review findings and added tests but did not resolve merge conflicts; the second properly merged main and added more coverage tests. The human approved and merged the same day with no additional comments beyond the review agent's findings. Review quality was strong — the medium finding (missing ErrTreeTruncated sentinel for truncated Git Trees API responses) was a real correctness issue that the code agent should have caught by replicating the existing truncation handling already present in the same file's commitFilesTo method. The 21-day idle period adds evidence to #966 (surface agent-approved PRs awaiting human review). The fix agent's two-iteration pattern adds evidence to #2728 (fix agent should verify explicit success criteria). Skipping a dedicated proposal for each since those issues already capture the pattern well. Proposals filed
|
Add forge.Client.ListRepositoryFiles to retrieve all file paths in a repository's default branch with a single Git Trees API call (refs → commit → tree?recursive=1). This replaces the O(N) GetFileContent pattern used by ComparePathPresence, reducing 100+ sequential API calls to 3 fixed calls regardless of path count.
Changes:
same refs/commits/trees pattern as CommitFiles)
calls ListRepositoryFiles once and checks membership locally
GetFileContent is never called; error injection and thread
safety coverage for the new forge method
PR #1954 introduces a naive ComparePathPresence in vendormanifest.go that loops GetFileContent per path. When that PR merges, its version should be replaced with this batch implementation.
Closes #2351
Post-script verification
agent/2351-batch-path-presence)32f73a4f93301493d2c31be3970aa4c51a26acc7..HEAD)