Skip to content

dashboard: in-process GitHub REST fetch with keep-alive (Closes #67) - #69

Merged
CameronCrow merged 2 commits into
mainfrom
feat/issue-67-github-rest-fetch
Jul 20, 2026
Merged

CameronCrow merged 2 commits into
mainfrom
feat/issue-67-github-rest-fetch

Conversation

@CameronCrow

Copy link
Copy Markdown
Owner

What & why

Every GitHub datum in packages/server-core/src/github-team.ts was fetched by spawning gh or git ls-remote, and each fresh process pays the full network-stack tax (WPAD + DNS + TLS) — 10-35s per spawn on Cameron's network. This replaces the transport with direct GitHub REST v3 calls over Node/Bun's global fetch, which pools and keeps connections alive by default, so the handshake is paid once per server lifetime.

Closes #67

Seam discipline (coordination with #66)

All changes are confined beneath the fetchTeamGitHubSnapshot boundary that #66 wraps with a TTL cache. fetchTeamGitHubSnapshot's signature is unchanged, no snapshot-level cache was added, and the per-worktree fetchGitHubPRStatus cache (CACHE_TTL_MS = 150_000, af80ddc) is kept as-is. A rebase over #66's wrapper should be mechanical.

Endpoint mapping

Old spawn New REST call
gh repo view git config --get remote.origin.url (local) parsed to owner/repo
gh pr view / gh pr list --search head: GET /repos/{o}/{r}/pulls?head={o}:{branch}&state=all + GET /commits/{sha}/check-runs (and combined /status)
git ls-remote --heads origin <branch> GET /repos/{o}/{r}/branches/{branch} (404 = false)
fetchTeamIssues GET /repos/{o}/{r}/issues?state=all&per_page=100 (PRs filtered out by pull_request key)
fetchTeamPRs GET /repos/{o}/{r}/pulls?state=all&per_page=50

Auth

gh auth token is run once, lazily, and cached in-process. On a 401 the token is invalidated, gh auth token re-run once, and the request retried a single time. Falls back to GITHUB_TOKEN/GH_TOKEN when gh is absent. No token → the same degraded empties the old code returned on gh failure.

Shape parity

  • Every exported signature and derived shape (deriveWorkBoard / deriveActivityFeed / roster) is identical.
  • Zod schemas adapted from gh camelCase to REST snake_case (headRefName → head.ref, statusCheckRollup → check-runs, etc.).
  • statusCheckRollup equivalence reproduced from check-runs + legacy combined statuses → the same checksStatus (failure > pending > success > none) and CheckItem[].
  • reviewDecision reproduced from GET /pulls/{n}/reviews (latest decisive review per user; changes-requested wins, else approved, else pending).
  • sharesAncestry's local git reads are kept unchanged (no network fallback existed to drop).
  • Team-PR check status is fetched only for open PRs (the only ones the board renders with checks) to stay within the fetch budget.

Verification

  • bun test packages/server-core/src/github-team.test.ts → 38 pass. Existing mocks needed no change (they exercise the pure derivations + the non-repo degraded-empty path); added REST→shape mapping coverage: parseRepoRef, restChecksToCheckItems/checksStatusFromItems (success/failure/pending rollups + skipped/legacy-status merge), computeReviewDecision, mapRestIssueToTeam (incl. PR-filtering), mapRestPRToTeam, formatWorktreePR.
  • bun run typecheck (server-core) → clean.
  • Full bun test server-core: the github-team suite is green; 14 pre-existing failures remain in unrelated files (terminal-host, agent bridges, askAgent, git-changes, scaffoldAgentMemory) — those source/test files are byte-identical to origin/main, so they're environment-dependent and untouched by this change.

<2s warm-fetch acceptance

A timing log gated on PAPYRUS_GH_TIMING wraps both fetchTeamGitHubSnapshot and fetchGitHubPRStatus ([github-team] ... <ms>). Manual check: run the server with PAPYRUS_GH_TIMING=1, open the team dashboard, and confirm the first refresh (cold connection) is slow while the second warm refresh logs well under 2000ms. I could not drive the live server from this worktree, so this is documented rather than captured — no measurement is fabricated.

🤖 Generated with Claude Code

CameronCrow and others added 2 commits July 20, 2026 15:59
…live)

Replace every `gh`/`git ls-remote` spawn in github-team.ts with direct
GitHub REST v3 calls over Node/Bun's global fetch, which pools and keeps
connections alive by default. This pays the DNS+TLS handshake once per
server lifetime instead of once per spawned process (10-35s each on
slow-DNS networks), so a warm dashboard refresh completes in well under 2s.

Transport changes are confined beneath the fetchTeamGitHubSnapshot
boundary (issue #66's cache seam) — that signature and all derived shapes
(deriveWorkBoard/deriveActivityFeed/roster) are untouched.

- Auth: `gh auth token` run once lazily and cached in-process; on 401 the
  token is refreshed once and the request retried. Falls back to
  GITHUB_TOKEN/GH_TOKEN when gh is absent. No token -> same degraded
  empties as before.
- Endpoint mapping: remote.origin.url (local) -> owner/repo; pulls?head=
  + commits/{sha}/check-runs (+ combined status) for PR status incl. a
  statusCheckRollup equivalent; branches/{branch} (404=false) for branch
  existence; issues?state=all (PRs filtered by pull_request key) and
  pulls?state=all for the team snapshot.
- Schemas adapted from gh camelCase to REST snake_case; sharesAncestry's
  local git reads kept as-is; per-worktree PR-status cache/TTL kept.
- Tests: existing suite passes; added REST->shape mapping coverage incl.
  statusCheckRollup success/failure/pending. A timing log gated on
  PAPYRUS_GH_TIMING backs the <2s warm-fetch acceptance check.

Closes #67

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@CameronCrow
CameronCrow force-pushed the feat/issue-67-github-rest-fetch branch from 5112b11 to a249764 Compare July 20, 2026 20:59
@CameronCrow
CameronCrow merged commit f632762 into main Jul 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant