Skip to content

docs(dependabot): a fine-grained token cannot read private check runs - #133

Merged
twistedmelonman merged 2 commits into
mainfrom
claude/digest-token-contents-dee4868f
Sep 12, 2026
Merged

docs(dependabot): a fine-grained token cannot read private check runs#133
twistedmelonman merged 2 commits into
mainfrom
claude/digest-token-contents-dee4868f

Conversation

@twistedmelonman

@twistedmelonman twistedmelonman commented Sep 12, 2026

Copy link
Copy Markdown
Member

The digest tokens need Contents and Commit statuses — and the digest still
cannot survey private repositories, because the last permission it needs does
not exist on a fine-grained token.

A fine-grained PAT has no Checks permission at all
(community#129512).
GitHub Actions results are check runs, so no scoping lets one read them on a
private repo. An earlier revision of this PR said to grant "Checks and Commit
statuses"; half of that was impossible. It is removed rather than softened.

Probed 2026-09-11 against nightowlstudiollc/kebab-tax-netlify#280 with all
four grantable permissions in place:

Endpoint Result
pulls/280 ok
commits/<sha> ok — Contents
commits/<sha>/status ok — Commit statuses
commits/<sha>/check-runs DENIED — ungrantable

The denial is silent, which is the danger. GitHub answers statusCheckRollup
with HTTP 200 and the correct totalCount, then nulls every CheckRun. That PR
has 12 contexts; 11 came back null and one Netlify StatusContext survived, so
seven failing builds and a green required check read as "1 failing check, 0
required checks". A PR with no Netlify status — most of the fleet — reads as
entirely green.

#134 is the code-side defence against that (collect refuses, classify buckets,
render warns). This PR is the documentation side: it records the finding and
states the remaining choice as an open decision rather than a step to follow.

Open decision, for @smartwatermelon:

Option Trade-off
Classic PAT Reads everything, but carries write access fleet-wide for a read-only report
GitHub App Apps do have a Checks permission; correct shape, more setup
Public-only Keep the current tokens; digest states plainly that private repos are unsurveyed

Until that is decided the workflow keeps failing on private repos rather than
publishing a partial digest.

Advances #120.

https://claude.ai/code/session_01ESsw699T54JHARkQXrdL3o

@twistedmelonman twistedmelonman changed the title docs(dependabot): the digest tokens need Contents: read docs(dependabot): the digest tokens need Contents: read (necessary, not sufficient) Sep 12, 2026
@twistedmelonman

Copy link
Copy Markdown
Member Author

Correction: Contents was necessary but not sufficient

The tokens were granted Contents: read and the run still fails with the
same FORBIDDEN on nightowlstudiollc/kebab-tax-netlify#280. The title
overstated what was established, and is now corrected.

What the grant did change, measured against the live token after the edit:

kebab-tax-netlify  private  commits=ok  pulls=ok
cleanroom          private  commits=ok  pulls=ok
amelia-boone       public   commits=ok  pulls=ok

So the REST reads all pass and the visibility theory is dead too. The runbook
change here stands on its own — the collector genuinely does read
pullRequest.commits, and Contents genuinely does gate it — but it is not the
whole answer.

What was never tested, and is the actual gap: the full pr_detail_query
has only ever run in CI. The local dry run used a broad personal login for all
three owners, and the probes above used REST plus a cut-down GraphQL query. The
real query adds statusCheckRollup, isRequired(pullRequestNumber:) and
checkSuite — and since smartwatermelon has no open PRs and
kebab-tax-netlify#280 sorts first under nightowlstudiollc, that query has
never once succeeded in CI under a fine-grained token. "It works on
amelia-boone" was a REST result, not a detail-query result.

A bisect of that query against the real token is running now. Holding this PR
until it names the permission, rather than merging a runbook that says "add
Contents" to someone who just did.

@twistedmelonman twistedmelonman changed the title docs(dependabot): the digest tokens need Contents: read (necessary, not sufficient) docs(dependabot): a fine-grained token cannot read private check runs Sep 12, 2026
@twistedmelonman

Copy link
Copy Markdown
Member Author

Resolved — the bisect completed and this PR was rewritten to match its result. The hold in my earlier comment is lifted.

What the bisect named. Probing the four REST endpoints against the live token on nightowlstudiollc/kebab-tax-netlify#280, with Contents and Commit statuses both granted:

pulls/280                  ok      <- Pull requests
commits/<sha>              ok      <- Contents
commits/<sha>/status       ok      <- Commit statuses
commits/<sha>/check-runs   DENIED  <- Checks

check-runs is the one that fails, and it is ungrantable: fine-grained PATs have no Checks permission at all (github.com/orgs/community/discussions/129512). That is why granting Contents did not fix it — Contents was necessary and never sufficient, and the missing piece was never a grant anyone could add.

This also corrected a claim I had made earlier in the thread: Commit statuses is grantable and now works. Only Checks is impossible.

Why the probes looked like they passed. They never exercised statusCheckRollup. GitHub does not refuse that field — it answers HTTP 200 with the correct totalCount and nulls every CheckRun. On that PR, 11 of 12 contexts came back null; only a Netlify StatusContext survived. Seven failing builds and one green required check presented as "1 failing check, 0 required checks".

The diff was rewritten, not just re-titled. The commit that said "needs Checks and Commit statuses too" was amended (it had never been pushed). The runbook no longer instructs anyone to grant Checks; it has a What a fine-grained token cannot do section stating the limit, the measurement, and the three ways forward as an open decision. The four-endpoint probe survives with its check-runs row corrected to say there is nothing to grant.

Superseded in part. #134 (merged) is the code-side defence: collect.sh refuses a rollup containing a nulled context, classify.sh buckets it checks-unreadable, render.sh warns. #135 plans the migration to a GitHub App, which does have a Checks permission. This PR is the documentation of the finding, and main's runbook is currently wrong without it — it still says "exactly two" permissions.

Claude Code Bot added 2 commits September 11, 2026 19:40
The runbook asked for Pull requests + Metadata. That is not enough, and the
gap is invisible until a run fails: the collector reads each PR's checks
through `pullRequest.commits(last:1)`, and GitHub gates the commit behind
Contents, not Pull requests.

Measured against the live token on 2026-09-11, after the first two runs failed:
`repos/<owner>/<repo>/pulls` returned 200 while
`repos/<owner>/<repo>/commits` returned 403 for the same token on the same
repo. The GraphQL error path said the same thing —
["repository","pullRequest","commits","nodes",0].

Two earlier hypotheses were wrong and are worth not repeating. It is not
Commit statuses: read — a probe read a StatusContext successfully on another
repo with the same token. It is not check-type-specific, not fork-related, and
not a context-count limit; all three were tested and ruled out.

One finding is not yet explained and is recorded as such: the same token reads
`commits` on one repo in the org and not another, though it was minted for
"All repositories". That pattern is per-repository approval rather than a
missing permission, so the troubleshooting table sends the reader to the org's
active-token list instead of re-minting with identical settings.

Also adds a FORBIDDEN troubleshooting section, because the collector stops at
the first failure — which means a later repo appearing to work is not evidence
that it does. It was never attempted.

Advances #120.

Claude-Session: https://claude.ai/code/session_01ESsw699T54JHARkQXrdL3o
Contents and Commit statuses were both necessary, and the digest still cannot
survey private repos, because the last permission it needs does not exist.

A fine-grained PAT has no Checks permission at all
(github.com/orgs/community/discussions/129512). GitHub Actions results are
check runs, so no amount of scoping lets one read them on a private
repository. The earlier claim in this file — grant Checks and Commit statuses
— was half right and half impossible; it is removed rather than softened.

Probed 2026-09-11 against nightowlstudiollc/kebab-tax-netlify#280 with all
four grantable permissions in place:

  pulls/280                  ok
  commits/<sha>              ok      <- Contents
  commits/<sha>/status       ok      <- Commit statuses
  commits/<sha>/check-runs   DENIED  <- ungrantable

The denial is silent, which is the danger. GitHub answers statusCheckRollup
with HTTP 200 and the correct totalCount, then nulls every CheckRun. That PR
has 12 contexts; 11 returned null and one Netlify StatusContext survived, so
seven failing builds and a green required check read as "1 failing check, 0
required checks". A PR with no Netlify status — most of the fleet — reads as
entirely green.

The runbook now states this as an open design decision rather than a step:
classic PAT (write access fleet-wide), GitHub App (which does have Checks), or
accepting public-repo-only coverage. Until it is decided the workflow keeps
failing on private repos rather than publishing a partial digest.

The four-endpoint probe is kept and corrected — it maps one-to-one onto the
permissions, and its check-runs row now says there is nothing to grant.

Advances #120.

Claude-Session: https://claude.ai/code/session_01ESsw699T54JHARkQXrdL3o
@twistedmelonman
twistedmelonman force-pushed the claude/digest-token-contents-dee4868f branch from 995a3be to b81adb7 Compare September 12, 2026 02:40
@twistedmelonman
twistedmelonman merged commit 4262ba6 into main Sep 12, 2026
3 checks passed
@twistedmelonman
twistedmelonman deleted the claude/digest-token-contents-dee4868f branch September 12, 2026 02:41
twistedmelonman pushed a commit that referenced this pull request Sep 12, 2026
#133 and #134 landed while this plan was in review, so three statements in it
were stale on arrival: the prerequisite line, step 4's "branched from main
after #134 merges", and an open item recommending #133 be closed unmerged.

Step 8 now rewrites a runbook that is accurate rather than one that is wrong —
it condenses the fine-grained material to a paragraph instead of correcting it.

Verified against origin/main rather than assumed: the nulled-checks guard is
present in collect.sh and the checks-unreadable bucket in classify.sh.

Claude-Session: https://claude.ai/code/session_01ESsw699T54JHARkQXrdL3o
twistedmelonman added a commit that referenced this pull request Sep 12, 2026
#135)

* docs(dependabot): plan the move from fine-grained PATs to a GitHub App

The three fine-grained tokens cannot do the job and no scoping fixes it: a
fine-grained PAT has no Checks permission at all
(github.com/orgs/community/discussions/129512), so it cannot read GitHub
Actions results on a private repository. Apps do have one, and a single app
installs on all three owners — which also removes the reason there are three
credentials.

The plan records the measurement behind the decision, the human steps (app
creation, private key, three installs, two secrets), the agent steps (workflow
minting via create-github-app-token, pinned to its v3.2.0 SHA), and the
cleanup, gated behind a verified run rather than listed before it.

Three things are called out as NOT fine-grained-token artifacts, because
"remove what the experiment added" would otherwise read as including them:

  - the three-layer nulled-checks guard (#134) — it enforces "unreadable is
    not absent", which holds for any credential, and a published digest is
    itself the proof the migration worked
  - verify_private_visibility and DIGEST_PROBE_* — an installation scoped to
    selected repositories degrades the same silent public-only way
  - run-digest.sh's per-owner DIGEST_TOKEN_<OWNER> indirection — it reads the
    token from the environment and does not care how it was obtained, which is
    what makes this migration workflow-only

collect.sh needs one edit, not a revert: its refusal message names fine-grained
tokens specifically and would misdirect if an app installation ever hit the
same guard.

STATUS.md now states the real blocker. It still said the digest was waiting on
tokens that have since been installed.

Advances #120.

Claude-Session: https://claude.ai/code/session_01ESsw699T54JHARkQXrdL3o

* docs(dependabot): the plan's prerequisites are merged, not pending

#133 and #134 landed while this plan was in review, so three statements in it
were stale on arrival: the prerequisite line, step 4's "branched from main
after #134 merges", and an open item recommending #133 be closed unmerged.

Step 8 now rewrites a runbook that is accurate rather than one that is wrong —
it condenses the fine-grained material to a paragraph instead of correcting it.

Verified against origin/main rather than assumed: the nulled-checks guard is
present in collect.sh and the checks-unreadable bucket in classify.sh.

Claude-Session: https://claude.ai/code/session_01ESsw699T54JHARkQXrdL3o

---------

Co-authored-by: Claude Code Bot <claude-code@smartwatermelon.github>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant