docs(dependabot): a fine-grained token cannot read private check runs - #133
Conversation
Correction: Contents was necessary but not sufficientThe tokens were granted Contents: read and the run still fails with the What the grant did change, measured against the live token after the edit: So the REST reads all pass and the visibility theory is dead too. The runbook What was never tested, and is the actual gap: the full A bisect of that query against the real token is running now. Holding this PR |
|
Resolved — the bisect completed and this PR was rewritten to match its result. The hold in my earlier comment is lifted. What the bisect named. Probing the four REST endpoints against the live token on
This also corrected a claim I had made earlier in the thread: Commit statuses is grantable and now works. Only Checks is impossible. Why the probes looked like they passed. They never exercised The diff was rewritten, not just re-titled. The commit that said "needs Checks and Commit statuses too" was amended (it had never been pushed). The runbook no longer instructs anyone to grant Checks; it has a What a fine-grained token cannot do section stating the limit, the measurement, and the three ways forward as an open decision. The four-endpoint probe survives with its Superseded in part. #134 (merged) is the code-side defence: collect.sh refuses a rollup containing a nulled context, classify.sh buckets it |
The runbook asked for Pull requests + Metadata. That is not enough, and the gap is invisible until a run fails: the collector reads each PR's checks through `pullRequest.commits(last:1)`, and GitHub gates the commit behind Contents, not Pull requests. Measured against the live token on 2026-09-11, after the first two runs failed: `repos/<owner>/<repo>/pulls` returned 200 while `repos/<owner>/<repo>/commits` returned 403 for the same token on the same repo. The GraphQL error path said the same thing — ["repository","pullRequest","commits","nodes",0]. Two earlier hypotheses were wrong and are worth not repeating. It is not Commit statuses: read — a probe read a StatusContext successfully on another repo with the same token. It is not check-type-specific, not fork-related, and not a context-count limit; all three were tested and ruled out. One finding is not yet explained and is recorded as such: the same token reads `commits` on one repo in the org and not another, though it was minted for "All repositories". That pattern is per-repository approval rather than a missing permission, so the troubleshooting table sends the reader to the org's active-token list instead of re-minting with identical settings. Also adds a FORBIDDEN troubleshooting section, because the collector stops at the first failure — which means a later repo appearing to work is not evidence that it does. It was never attempted. Advances #120. Claude-Session: https://claude.ai/code/session_01ESsw699T54JHARkQXrdL3o
Contents and Commit statuses were both necessary, and the digest still cannot survey private repos, because the last permission it needs does not exist. A fine-grained PAT has no Checks permission at all (github.com/orgs/community/discussions/129512). GitHub Actions results are check runs, so no amount of scoping lets one read them on a private repository. The earlier claim in this file — grant Checks and Commit statuses — was half right and half impossible; it is removed rather than softened. Probed 2026-09-11 against nightowlstudiollc/kebab-tax-netlify#280 with all four grantable permissions in place: pulls/280 ok commits/<sha> ok <- Contents commits/<sha>/status ok <- Commit statuses commits/<sha>/check-runs DENIED <- ungrantable The denial is silent, which is the danger. GitHub answers statusCheckRollup with HTTP 200 and the correct totalCount, then nulls every CheckRun. That PR has 12 contexts; 11 returned null and one Netlify StatusContext survived, so seven failing builds and a green required check read as "1 failing check, 0 required checks". A PR with no Netlify status — most of the fleet — reads as entirely green. The runbook now states this as an open design decision rather than a step: classic PAT (write access fleet-wide), GitHub App (which does have Checks), or accepting public-repo-only coverage. Until it is decided the workflow keeps failing on private repos rather than publishing a partial digest. The four-endpoint probe is kept and corrected — it maps one-to-one onto the permissions, and its check-runs row now says there is nothing to grant. Advances #120. Claude-Session: https://claude.ai/code/session_01ESsw699T54JHARkQXrdL3o
995a3be to
b81adb7
Compare
#133 and #134 landed while this plan was in review, so three statements in it were stale on arrival: the prerequisite line, step 4's "branched from main after #134 merges", and an open item recommending #133 be closed unmerged. Step 8 now rewrites a runbook that is accurate rather than one that is wrong — it condenses the fine-grained material to a paragraph instead of correcting it. Verified against origin/main rather than assumed: the nulled-checks guard is present in collect.sh and the checks-unreadable bucket in classify.sh. Claude-Session: https://claude.ai/code/session_01ESsw699T54JHARkQXrdL3o
#135) * docs(dependabot): plan the move from fine-grained PATs to a GitHub App The three fine-grained tokens cannot do the job and no scoping fixes it: a fine-grained PAT has no Checks permission at all (github.com/orgs/community/discussions/129512), so it cannot read GitHub Actions results on a private repository. Apps do have one, and a single app installs on all three owners — which also removes the reason there are three credentials. The plan records the measurement behind the decision, the human steps (app creation, private key, three installs, two secrets), the agent steps (workflow minting via create-github-app-token, pinned to its v3.2.0 SHA), and the cleanup, gated behind a verified run rather than listed before it. Three things are called out as NOT fine-grained-token artifacts, because "remove what the experiment added" would otherwise read as including them: - the three-layer nulled-checks guard (#134) — it enforces "unreadable is not absent", which holds for any credential, and a published digest is itself the proof the migration worked - verify_private_visibility and DIGEST_PROBE_* — an installation scoped to selected repositories degrades the same silent public-only way - run-digest.sh's per-owner DIGEST_TOKEN_<OWNER> indirection — it reads the token from the environment and does not care how it was obtained, which is what makes this migration workflow-only collect.sh needs one edit, not a revert: its refusal message names fine-grained tokens specifically and would misdirect if an app installation ever hit the same guard. STATUS.md now states the real blocker. It still said the digest was waiting on tokens that have since been installed. Advances #120. Claude-Session: https://claude.ai/code/session_01ESsw699T54JHARkQXrdL3o * docs(dependabot): the plan's prerequisites are merged, not pending #133 and #134 landed while this plan was in review, so three statements in it were stale on arrival: the prerequisite line, step 4's "branched from main after #134 merges", and an open item recommending #133 be closed unmerged. Step 8 now rewrites a runbook that is accurate rather than one that is wrong — it condenses the fine-grained material to a paragraph instead of correcting it. Verified against origin/main rather than assumed: the nulled-checks guard is present in collect.sh and the checks-unreadable bucket in classify.sh. Claude-Session: https://claude.ai/code/session_01ESsw699T54JHARkQXrdL3o --------- Co-authored-by: Claude Code Bot <claude-code@smartwatermelon.github>
The digest tokens need Contents and Commit statuses — and the digest still
cannot survey private repositories, because the last permission it needs does
not exist on a fine-grained token.
A fine-grained PAT has no Checks permission at all
(community#129512).
GitHub Actions results are check runs, so no scoping lets one read them on a
private repo. An earlier revision of this PR said to grant "Checks and Commit
statuses"; half of that was impossible. It is removed rather than softened.
Probed 2026-09-11 against
nightowlstudiollc/kebab-tax-netlify#280with allfour grantable permissions in place:
pulls/280commits/<sha>commits/<sha>/statuscommits/<sha>/check-runsThe denial is silent, which is the danger. GitHub answers
statusCheckRollupwith HTTP 200 and the correct
totalCount, then nulls every CheckRun. That PRhas 12 contexts; 11 came back null and one Netlify StatusContext survived, so
seven failing builds and a green required check read as "1 failing check, 0
required checks". A PR with no Netlify status — most of the fleet — reads as
entirely green.
#134 is the code-side defence against that (collect refuses, classify buckets,
render warns). This PR is the documentation side: it records the finding and
states the remaining choice as an open decision rather than a step to follow.
Open decision, for @smartwatermelon:
Until that is decided the workflow keeps failing on private repos rather than
publishing a partial digest.
Advances #120.
https://claude.ai/code/session_01ESsw699T54JHARkQXrdL3o