feat(ci): gate on required status contexts nothing can publish by hyperpolymath · Pull Request #869 · hyperpolymath/standards

hyperpolymath · 2026-09-20T21:53:19Z

What

Adds a gate for a class of rule that is invisible from the CI side: a required status context that nothing can publish.

GitHub matches a required status check by string equality against the name it publishes for that check run, and never checks that the name is producible. A rule requiring a name no publisher can supply is unsatisfiable — every pull request stays BLOCKED, the board shows green, and no CI output names the cause.

This is the general form of hyperpolymath/tropical-types#17 (bare Hypatia Neurosymbolic Analysis required while the wrapper published scan / Hypatia Neurosymbolic Analysis) and of the wrapper-prefix mismatch this estate already documented in docs/audits/audit-hypatia-pin-orphan-2026-05-27.adoc.

File	What it is
`scripts/check-required-contexts.sh`	The gate: derives publishable names, reads required contexts, reports the residue
`scripts/tests/check-required-contexts-test.sh`	Hermetic fixture suite (no network, no token, no API)
`docs/audits/audit-ci-context-producibility-2026-09-20.adoc`	The audit that produced the gate, with the estate census

Read-only; bash + awk + jq + gh + curl, no new dependency.

Rules it encodes

a plain job publishes its name: — or <job id> (<matrix values>) when it has a matrix and no name: — or the job id alone;
a job that calls a reusable workflow publishes <caller display name or job id> / <inner job display name or job id>, so a wrapper can never publish the reusable's bare inner name;
a context bound to a third-party app integration is that app's to publish, not the repository's — skipped by design.

Behaviour

advisory by default (REQUIRED_CONTEXTS_STRICT=1 to gate) — a check that reds nobody yet is a warning, not a red;
an API failure (rate limit, missing scope) reports not inspected, never "nothing is required": a gate that fails open on the failure of the call it depends on is worse than no gate;
--print-commands emits the remediation, --json FILE emits a machine-readable report, --no-network/REUSABLE_CACHE_DIR make it hermetic.

Census (2026-09-20, all 365 non-archived public repositories)

337 required contexts: 168 app-owned, 156 producible, 4 app-named, 5 unsatisfiable across 4 repositories —

Repository	Ruleset	Required context
`awesome-gleam`	Base	`Adjust Configuration`, `Analyze (actions)`
`casket-ssg`	Optimus-Branch	`Analyze Code (actions)`
`coord-tui`	Optimus-Branch	`CodeQL Analysis (actions, none)`
`vext`	Optimus-Branch	`CodeQL Analysis (actions, none)`

Every one is a retired CodeQL-era job name. The same audit records the correct way to satisfy a bare-name requirement (the defiant inline-implementation precedent), the caller-id drift across 208 wrappers (200 scan vs 8 hypatia) against the canonical string already documented, and the caveat that legacy branch protection needs Administration: read to inspect.

Deliberately not included

Wiring this into governance-reusable.yml. That changes the check surface of every repository pinned to it; the audit records the shape and the advisory-first rationale for the owner to arm when ready.

Adjacent finding

Hypatia's research_extensions: RE001 asks jobs that reference the secrets context to install step-security/harden-runner — which is not in rhodium-standard-repositories/actions-allowlist/allowed-actions.json (89 patterns, no step-security/*). A repository under the allowlist cannot satisfy RE001 by the route the rule suggests. Either the rule or the allowlist should move; meanwhile the workable answer is github.token (as tropical-types now does) or routing the credentialed work through a reusable with secrets: inherit.

Verification

bash scripts/tests/check-required-contexts-test.sh — 5/5 branches pass.
bash scripts/check-canonical-names.sh origin/main — clean.
Census reproducible from the audit's method section; the gate was run against a live repository checkout during development (producible vs unsatisfiable paths both exercised).

GitHub matches a required status check by string equality against the name it publishes for that run, and never checks that the name is producible. A rule requiring a name no publisher can supply is unsatisfiable: every pull request stays BLOCKED while the CI board shows green, and nothing on the CI side names the cause. That is the general form of hyperpolymath/tropical-types#17 (the bare `Hypatia Neurosymbolic Analysis` required while the wrapper published `scan / Hypatia Neurosymbolic Analysis`) and of the wrapper-prefix mismatch recorded in docs/audits/audit-hypatia-pin-orphan-2026-05-27.adoc. Adds, all read-only and dependency-free (bash + awk + jq + gh + curl): * scripts/check-required-contexts.sh — derives the names a repository's workflows can publish (job display names, `<job id> (*)` matrix forms, and `<caller> / <inner job>` for reusable-wrapper callers), reads the required contexts from every enabled branch ruleset (and legacy branch protection when the credential can read it), and reports any required context that neither a workflow nor a bound app integration can satisfy. App-bound contexts are skipped by design: CodeRabbit, SonarCloud and the code-scanning app own their names. Advisory by default (REQUIRED_CONTEXTS_STRICT=1 to gate), and an API failure is reported as "not inspected" rather than "nothing is required" — a gate that fails open on the failure of the call it depends on is worse than no gate. * scripts/tests/check-required-contexts-test.sh — drives the built-in hermetic fixture: a synthetic repository, a stubbed reusable in a cache directory, and JSON required-context sets. No network, no token, no API. Five branches, including a regression case for exact-versus-prefix matching. * docs/audits/audit-ci-context-producibility-2026-09-20.adoc — the audit that produced the gate. Census of all 365 non-archived public repositories: 337 required contexts, of which 168 are app-owned, 156 producible, 4 app-named and 5 unsatisfiable across 4 repositories (awesome-gleam, casket-ssg, coord-tui, vext — every one a retired CodeQL-era job name). Records the tropical-types case study, the two honest remediations for a bare-name requirement (with the defiant inline-implementation precedent), the caller-id drift across the estate (200 `scan` versus 8 `hypatia`), the caveats (legacy branch protection needs Administration: read; matrix forms are matched as a family), and the adjacent RE001-versus-allowlist inconsistency (step-security/harden-runner is not in the allowed-actions list the same estate enforces). Deliberately not wired into governance-reusable.yml: that changes the check surface of every repository pinned to it, and the audit records the shape and the advisory-first rationale instead.

coderabbitai · 2026-09-20T21:53:31Z

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 61084e1b-72cb-42ce-8279-dc9c3e3055b1

📥 Commits

Reviewing files that changed from the base of the PR and between 6907433 and f2146fa.

📒 Files selected for processing (3)

docs/audits/audit-ci-context-producibility-2026-09-20.adoc
scripts/check-required-contexts.sh
scripts/tests/check-required-contexts-test.sh

 _______________________________
< Yippee-ki-yay, mother-bugger! >
 -------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       / 　 づ

✨ Finishing Touches

📝 Generate docstrings

Commit to this branch
Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

sonarqubecloud · 2026-09-20T21:54:02Z

Quality Gate failed

Failed conditions
C Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

…in the gate header (#871) ## What Two self-inflicted defects in the artefacts that merged as #869, both caught by re-reading them against the census data they quote. **1. The audit miscounted its own census.** It said *337 required contexts across 179 repositories*; the verdicts it prints alongside (app-owned 168 + producible 156 + app-named 4 + unsatisfiable 5) total **333**. Corrected, plus the fact that all 333 were ruleset-sourced, so no repository in the census still enforced required status checks through legacy branch protection. **2. The gate's header described `hyperpolymath/tropical-types#17` in the present tense** ("the wrapper publishes `scan / Hypatia Neurosymbolic Analysis`"). That repository publishes the bare name from an inline job and the prefixed name from its wrapper caller now — the sentence was stale on merge. Also records why the five unsatisfiable contexts matter rather than being app-configuration work: every one binds `integration_id: 15368` (GitHub Actions), so they are repository-owned workflow job names. No external publisher could ever satisfy them, and the fix is a rename or a re-armed rule, not a settings tweak. ## Why it is worth a PR rather than a quiet push The whole point of this audit is that unverified claims about CI state are how a required context becomes unsatisfiable without anyone noticing. A census document that miscounts its own census, and a header that describes a fixed condition as live, are the same failure at one remove. ## Verification * `bash scripts/tests/check-required-contexts-test.sh` — 5/5 pass (unchanged by the edit; run to confirm nothing else moved). * `bash -n scripts/check-required-contexts.sh` — parses. * Counts cross-checked against the census records: 333 contexts over 179 repositories, 5 unsatisfiable in 4 repositories (`awesome-gleam` ×2, `casket-ssg`, `coord-tui`, `vext`), all `integration_id: 15368`.

hyperpolymath merged commit a087ce5 into main Sep 20, 2026
22 of 30 checks passed

hyperpolymath deleted the fix/required-contexts-producibility branch September 20, 2026 21:57

hyperpolymath mentioned this pull request Sep 20, 2026

fix(audit): correct the census total to 333, and the stale #17 tense in the gate header #871

Merged

Navigation Menu

Uh oh!

feat(ci): gate on required status contexts nothing can publish - #869

feat(ci): gate on required status contexts nothing can publish#869
hyperpolymath merged 1 commit into
mainfrom
fix/required-contexts-producibility

hyperpolymath commented Sep 20, 2026

Uh oh!

coderabbitai Bot commented Sep 20, 2026

Uh oh!

sonarqubecloud Bot commented Sep 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

hyperpolymath commented Sep 20, 2026

What

Contents

Rules it encodes

Behaviour

Census (2026-09-20, all 365 non-archived public repositories)

Deliberately not included

Adjacent finding

Verification

Uh oh!

coderabbitai Bot commented Sep 20, 2026

Uh oh!

sonarqubecloud Bot commented Sep 20, 2026

Quality Gate failed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant