diff --git a/.github/workflows/dependabot-digest.yml b/.github/workflows/dependabot-digest.yml index abb6192..c996aba 100644 --- a/.github/workflows/dependabot-digest.yml +++ b/.github/workflows/dependabot-digest.yml @@ -50,12 +50,22 @@ jobs: env: # One token per owner: a fine-grained PAT is "limited to access # resources owned by a single user or organization" (GitHub docs), so - # three owners need three tokens. Each needs only Pull requests: read - # and Metadata: read, on all repositories for that owner. + # three owners need three tokens. Each needs Pull requests, Contents, + # Commit statuses and Metadata — all read-only, on all repositories + # for that owner. # - # The alternative, a single classic PAT, would carry write access to - # every repository in every org — far more than a read-only digest - # should hold. + # Those four are the most a fine-grained token can be given, and they + # are NOT enough to survey private repos: a fine-grained PAT has no + # Checks permission at all + # (github.com/orgs/community/discussions/129512), so it cannot read + # GitHub Actions results there. GitHub does not error — it returns + # statusCheckRollup with HTTP 200 and nulls every CheckRun, which + # would turn failing builds into "nothing failing". collect.sh + # refuses such a response rather than under-report. + # + # Resolving that needs a classic PAT (write access fleet-wide), a + # GitHub App (which does have Checks), or accepting public-repo-only + # coverage. See docs/runbooks/dependabot-digest-tokens.md. DIGEST_TOKEN_SMARTWATERMELON: ${{ secrets.DIGEST_TOKEN_SMARTWATERMELON }} DIGEST_TOKEN_NIGHTOWLSTUDIOLLC: ${{ secrets.DIGEST_TOKEN_NIGHTOWLSTUDIOLLC }} DIGEST_TOKEN_TWISTEDMELONMAN: ${{ secrets.DIGEST_TOKEN_TWISTEDMELONMAN }} diff --git a/docs/runbooks/dependabot-digest-tokens.md b/docs/runbooks/dependabot-digest-tokens.md index c90d35d..d78f319 100644 --- a/docs/runbooks/dependabot-digest-tokens.md +++ b/docs/runbooks/dependabot-digest-tokens.md @@ -41,12 +41,78 @@ For each of the three owners, at installing* below. 4. **Repository access**: **All repositories**. The digest must see every repo under the owner, including ones created after the token was minted. -5. **Repository permissions** — exactly two, both read-only: +5. **Repository permissions** — all four, all read-only: - **Pull requests**: Read-only + - **Contents**: Read-only + - **Commit statuses**: Read-only - **Metadata**: Read-only (mandatory; GitHub selects it automatically) Grant nothing else. The digest never writes to any surveyed repository. + + **These four are the most a fine-grained token can be given, and they are + not enough to run the digest on private repositories.** Read the next + section before minting anything. + + The collector reads each PR's checks through + `pullRequest.commits(last:1).commit.statusCheckRollup`. Each hop needs its + own permission, and `statusCheckRollup` merges two REST resources: + `commits//status` (**Commit statuses**), which these four grants do + cover, and `commits//check-runs` (**Checks**), which they cannot — + see below. 6. For the two org tokens, the request may need org approval before it works. + Approval can be **per repository**: a token minted for "All repositories" + can still answer for some repos and return 403 for others, which looks + exactly like a permissions error and is not one. If one repo fails while + its neighbours succeed, check the org's Settings → Personal access tokens → + Active tokens for that token's actual repository list before re-minting. + +## What a fine-grained token cannot do + +**A fine-grained PAT has no Checks permission.** It is not a grant that was +overlooked; the permission does not exist on the fine-grained list at all +(). Since GitHub Actions +results are check runs, a fine-grained token cannot read them on a private +repository no matter how it is scoped. + +The failure is silent, which is what makes it dangerous. GitHub does not +return an error for the unreadable part. It answers `statusCheckRollup` with +HTTP 200 and the correct `totalCount`, then sets every CheckRun's fields to +null. Measured 2026-09-11 on `nightowlstudiollc/kebab-tax-netlify#280`, with +all four permissions above granted: + +| Endpoint | Result | +| --- | --- | +| `pulls/280` | ok | +| `commits/` | ok | +| `commits//status` | ok — this is why one Netlify status survived | +| `commits//check-runs` | **DENIED — and ungrantable** | + +That PR has 12 contexts. Eleven came back null and one Netlify StatusContext +survived. Mapped naively, seven failing builds and one green required check +read as "1 failing check, 0 required checks". A PR with no Netlify status at +all — most of the fleet — reads as entirely green and lands in +`ready-to-merge` with a paste-ready merge-lock line under it. + +`collect.sh` therefore refuses any rollup containing a nulled context rather +than publishing an under-reporting digest, `classify.sh` buckets such a PR as +`checks-unreadable`, and `render.sh` says so in the body. The digest fails +loudly instead of quietly telling you to merge broken code. + +Public repositories are unaffected: check results there are readable without +Checks or Commit statuses at all. That is why an under-scoped token appears to +work until it meets a private repo — and why testing against a public repo +proves nothing. + +**This is an open design decision, not a step to follow.** Three ways forward: + +| Option | Trade-off | +| --- | --- | +| Classic PAT | One token reads everything, including check runs. But it carries write access across every repo in both orgs for a read-only report. | +| GitHub App | Apps *do* have a Checks permission. Correct scoping and the right long-term shape; more setup than a PAT. | +| Public-only coverage | Keep the fine-grained tokens and have the digest state plainly that private repos are unsurveyed. Honest, and incomplete. | + +Until that is decided, the workflow will keep failing on the private repos +rather than publishing a partial digest. ## Install the secrets @@ -113,6 +179,62 @@ scripts did not crash; only the diff shows the workflow saw the same fleet you can see. The collector's private-repo probe catches a token that lost private access, but it cannot catch a token that was scoped to the wrong owner. +## If a run fails with FORBIDDEN + +The run log names the repo, the PR, and GitHub's own reason: + +``` +collect.sh: nightowlstudiollc/kebab-tax-netlify#280: could not read PR detail (exit 1) +collect.sh: stderr: gh: Resource not accessible by personal access token +collect.sh: graphql: FORBIDDEN: Resource not accessible by personal access token +``` + +The collector stops at the first failure and publishes nothing, so a later repo +succeeding is **not** evidence its access is fine — it was never attempted. +Diagnose by probing the token directly rather than reasoning from which repos +appear to work. + +The tokens live in 1Password (vault `Automation`, item +`DIGEST_TOKEN_`, field `token`), so a probe can read one directly +instead of pasting it. Run against the **private** repo that failed: + +```bash +t="$(op read "op://Automation/DIGEST_TOKEN_NIGHTOWLSTUDIOLLC/token")" +sha="$(GH_TOKEN="$t" gh api repos/OWNER/REPO/pulls/NNN --jq '.head.sha')" +for path in \ + "pulls/NNN" \ + "commits/${sha}" \ + "commits/${sha}/check-runs" \ + "commits/${sha}/status" +do + if GH_TOKEN="$t" gh api "repos/OWNER/REPO/${path}" >/dev/null 2>&1; then + echo " ${path##*/}: ok" + else + echo " ${path##*/}: DENIED" + fi +done +unset t +``` + +Each line maps to exactly one permission: + +| Denied endpoint | Missing permission | +| --- | --- | +| `pulls/NNN` | **Pull requests: Read-only** | +| `commits/` | **Contents: Read-only** | +| `commits//status` | **Commit statuses: Read-only** | +| `commits//check-runs` | **Checks: Read-only** — which a fine-grained token cannot be given. On one, this line reads DENIED on every private repo and no grant changes it. See *What a fine-grained token cannot do*. | + +Two traps, both of which cost time on 2026-09-11: + +- **Test a private repo.** Check results on public repositories read without + Checks or Commit statuses, so a probe against a public repo passes with an + under-scoped token and proves nothing. +- **Editing a token's permissions does not change its value**, so the stored + secret stays valid and needs no reinstall. If a run still fails after a + grant change, re-run the probe before suspecting the secret — the token + string is almost certainly fine. + ## After installing Add a row per token to `docs/token-rotation.md` with its expiry, and set one