Skip to content

fix(audit,gate): withdraw the five residues — a file-based census cannot see GitHub-generated checks #876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hyperpolymath merged 1 commit into from
Sep 20, 2026
Merged

fix(audit,gate): withdraw the five residues — a file-based census cannot see GitHub-generated checks #876

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 40 additions & 19 deletions docs/audits/audit-ci-context-producibility-2026-09-20.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,16 +86,38 @@ status checks through legacy branch protection):
| 4
| GitHub-app-published names that carry no integration id (`github-advanced-security`, `Dependabot`)

| *unsatisfiable*
| *5*
| no workflow here publishes it and no app integration is bound to it
| unsatisfiable
| *0*
| see below — the five candidates this pass flagged were refuted by check-run history
|===

That is a good result for the estate and a poor one for the five rules: the class
is rare, which is exactly why it hides. All five carry `integration_id: 15368`
(GitHub Actions), so they are repository-owned *workflow job names* rather than
third-party app names — nothing outside the repository could ever publish them.
The residues, all in auxiliary (`Optimus-Branch`, `Base`) rulesets:
The file-based pass flagged five required contexts as candidates. **All five were
refuted** when put to the check-run history on 2026-09-20: each one reports on the
repository in question, published by a *GitHub-generated* workflow (code scanning
default setup and friends) that no file in the repository mentions:

[cols="1,1,1,1"]
|===
| Repository | Candidate context | Evidence in check-run history | Publisher

| `awesome-gleam` | `Analyze (actions)` | reports on 8 of the last 8 commits, and on PR #12's head | GitHub code scanning (dynamic)
| `awesome-gleam` | `Adjust Configuration` | reports on `main` (intermittent) | GitHub code scanning (dynamic)
| `casket-ssg` | `Analyze Code (actions)` | reports on 15 of the last 30 commits | GitHub code scanning (dynamic)
| `coord-tui` | `CodeQL Analysis (actions, none)` | reports on 5 of the last 8 commits, and on PR #63's head | GitHub code scanning (dynamic)
| `vext` | `CodeQL Analysis (actions, none)` | reports 12 times across the last 8 commits, and on PR #51's head | GitHub code scanning (dynamic)
|===

That is the finding this audit exists to publish: **a census over repository files
cannot see the checks GitHub generates itself.** The rule that follows is
operational, not cosmetic —

* a file-based pass may only produce *candidates*;
* an `unsatisfiable` verdict becomes a finding only after the check-run history of
the default branch confirms the name never reports;
* and a required context is never deleted on a file-based verdict alone.

The five entries previously listed here as live defects, in four auxiliary
(`Optimus-Branch`, `Base`) rulesets, were withdrawn for exactly that reason:

[cols="2,1,2,2"]
|===
Expand All @@ -108,17 +130,16 @@ The residues, all in auxiliary (`Optimus-Branch`, `Base`) rulesets:
| `hyperpolymath/vext` | Optimus-Branch | `CodeQL Analysis (actions, none)` | stale job-name + matrix tuple
|===

Each one was re-checked per ruleset on 2026-09-20 and is a live defect, not an
artefact of evaluating the wrong ref: every carrying ruleset is `active`, every
one binds `~DEFAULT_BRANCH` (these repositories have a single branch,
`main`), and every context carries `integration_id: 15368`. Note the naming
irony in three of them — a ruleset called `Optimus-Branch` guarding the default
branch.

Each one is the same defect as tropical-types#17 in a different costume: the
requirement names a check that used to exist. The fix is one of the two honest
options below; the diagnostic is
`scripts/check-required-contexts.sh --print-commands`.
The per-ruleset check was correct as far as it went — every carrying ruleset is
`active`, every one binds `~DEFAULT_BRANCH`, and every context carries
`integration_id: 15368` — but it answered the wrong question: whether the *rule*
was live, not whether a *publisher* exists. The publisher is a GitHub-generated
workflow, invisible to the repository and to this census.

The revised method, and the cost of getting it wrong, are the same lesson as
tropical-types#17: a name that nothing in the repository publishes can still be
satisfied by the platform, and a name that nothing publishes *anywhere* is a
blocked pull request nobody can diagnose from the CI board.

== Case study — tropical-types#17

Expand Down
17 changes: 15 additions & 2 deletions scripts/check-required-contexts.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,15 @@
# bare one), and catalogued for the wrapper case in
# docs/audits/audit-hypatia-pin-orphan-2026-05-27.adoc.
#
# LIMIT — READ THIS BEFORE ACTING ON A VERDICT
# --------------------------------------------
# A verdict here is derived from the repository's files. GitHub also generates
# workflows server-side (code scanning default setup and friends) and those
# publish check names nothing in the repository mentions. A name this script
# calls unsatisfiable may therefore be reporting on every commit. Confirm any
# such verdict against the check-run history of the default branch, and never
# delete a required context on this script's word alone.
#
# THE RULE THIS ENCODES
# ---------------------
# * a plain job publishes its `name:` — or `<job id> (<matrix values>)` when it
Expand Down Expand Up @@ -239,16 +248,20 @@ run_audit() {
UNSATISFIABLE)
unsatisfiable=$((unsatisfiable+1))
if [ "$STRICT" = 1 ]; then
err "required context '$ctx' ($src) is not producible by any workflow here, is not bound to an app integration, and no job publishes a matching name"
err "required context '$ctx' ($src) is not producible by any workflow here, is not bound to an app integration, and no job publishes a matching name (candidate only: GitHub-generated workflows publish checks no file mentions — confirm against the check-run history before changing the rule)"
else
warn "required context '$ctx' ($src) is not producible by any workflow here and is not bound to an app integration — unsatisfiable as written"
warn "required context '$ctx' ($src) is not producible by any workflow here and is not bound to an app integration — unsatisfiable as written (candidate only: GitHub-generated workflows publish checks no file mentions — confirm against the check-run history before changing the rule)"
fi
if [ "$PRINT_COMMANDS" = 1 ]; then
cat <<EOS
# Remediation — require a name this repository publishes, or publish this one:
# a) replace the requirement with a producible name (list above), or
# b) implement the check in-repo and publish '$ctx' from a plain job
# (a reusable-wrapper caller cannot: it publishes '<caller> / <inner job>').
# FIRST, though: confirm the name never reports. Read the check-run history of
# the default branch — GitHub-generated workflows publish checks that no file
# in this repository mentions, and five such candidates were refuted this way
# on 2026-09-20 (docs/audits/audit-ci-context-producibility-2026-09-20.adoc).
EOS
fi ;;
app-owned) info "app-owned ($src): $ctx" ;;
Expand Down
Loading