[estate] audit Base ruleset across repos for unsatisfiable rules that force bypass merges #473

Activity

1. hyperpolymath commented on Jun 13, 2026

   

   hyperpolymath

   on Jun 13, 2026

   OwnerAuthor

   More actions

   Governing principle for this audit (process/product plane split)

   This audit now has a conceptual backbone. The bypass-forcing rules all share one defect: a gate (merge admission) asserting a product outcome. The rule to apply repo-by-repo:

   A gate may assert only what is true of the candidate change at decision time. Outcomes are reconciled out-of-band; they never block admission. Forbidden edge: Gate → Outcome.

   Mechanically lintable (turns this from manual audit → recurring check)

   Flag any active ruleset rule whose satisfaction depends on post-merge or repo-wide state:

   • required_deployments (deploy is post-merge)
   • code_scanning as a rule where the tool returns neutral on non-code PRs (CodeQL) → move to required_status_checks contexts instead (neutral-tolerant)
   • code_scanning requiring a tool that uploads 0 analyses (e.g. Scorecard / startup_failure)
   • copilot_code_review / code_quality unless those features actually produce results
   • any rule satisfied by repo-wide state (e.g. "zero open alerts") rather than diff-scoped state

   Triage signal per repo

   gh api repos/OWNER/REPO/rulesets/rule-suites — if every recent entry is result: bypass, the ruleset is decorative (every merge bypassed). Compare before/after remediation; target a genuine result: pass.

   Caveats (carry these into every repo)

   • Owner sign-off per repo — this is governance config; do not blanket-change.
   • Owner pushes always show bypass (RepositoryRole bypass actor), so you cannot read per-rule pass/fail from owner merges — probe with a real green PR's mergeStateStatus.
   • Diff-scoped product checks (e.g. CodeQL-on-changed-lines) are legal gates; only repo-wide / post-merge assertions move to reconciliation.

   Full write-up + alternative mental models (control/data plane, hermetic-build, ToC; why supply-chain is a partial fit): dev-notes/2026-06-13-process-product-plane-split.adoc. Companion board change: Project 35 now carries a Plane field (Process/Product/Bridge).
2. hyperpolymath commented on Jul 3, 2026

   

   hyperpolymath

   on Jul 3, 2026

   OwnerAuthor

   More actions

   Production recurrence confirmed (2026-07-03)

   The code_scanning unsatisfiable-rule pattern from this audit (items 2–3) is still live and actively deadlocking PRs on estate repos. Verified today via the rulesets API:

   Repo	Base ruleset id	code_scanning_tools
   odds-and-sods-package-manager	14285488	CodeQL only (already remediated)
   standards	14285635	CodeQL + Hypatia + Scorecard ⟵ deadlocked
   rsr-template-repo	10892905	CodeQL + Hypatia + Scorecard ⟵ deadlocked

   Any PR on the two lagging repos sticks on "Waiting for code scanning results from Hypatia/Scorecard for the commit …" — Hypatia is gated via a status-check context (not a per-PR code-scanning upload) and Scorecard uploads 0 analyses, so neither can ever satisfy a code_scanning rule. This matches the mechanism already fixed on hypatia in #470.

   Root-cause note (source is already clean): the canonical provisioner hyperpolymath/git-scripts → scripts/branch-protection-apply.sh builds the Base ruleset with a code_scanning rule listing CodeQL only (thresholds errors/high_or_higher), and its git history never contained Hypatia/Scorecard. The live bad rulesets carry thresholds all/all, so they predate the current script and are stale live state, not a bad template. Re-running that script estate-wide (it PUTs Base in place) would itself remediate the residue. Neither rsr-template-repo nor standards defines rulesets in-tree, and rulesets are not copied by template instantiation — so there is no template file to fix.

   A parallel remediation pass is trimming the affected live rulesets to CodeQL-only now; this comment is to (a) confirm the recurrence for the estate audit and (b) record that the durable guardrail already exists in git-scripts (keep the provisioner CodeQL-only; gate Hypatia via required_status_checks contexts as done in #470). No new issue filed — this is fully covered by #470 + #473.
3. hyperpolymath

   added

   ci

   tri/controlSafety triangle — gate it so it cannot regressSafety triangle — gate it so it cannot regress

   on Aug 3, 2026
4. hyperpolymath

   added

   cicdCI/CD: workflows, actions, lockfiles, pins, runners, release gatesCI/CD: workflows, actions, lockfiles, pins, runners, release gates

   and removed

   ci

   on Aug 26, 2026
5. hyperpolymath

   added

   priority:p1High - schedule nextHigh - schedule next

   scope:estateAffects many or all repos across the estateAffects many or all repos across the estate

   status:readyFully specified and ready to be picked upFully specified and ready to be picked up

   on Sep 30, 2026
6. hyperpolymath commented on Sep 30, 2026

   

   hyperpolymath

   on Sep 30, 2026

   OwnerAuthor

   More actions

   Status (2026-09-30 issue sweep): reviewed and kept open as a live meta-tracker. Feature/RFC issues were consolidated into the roadmap #886. Root-cause fixes in flight: #883 (scanner build + npx precision), #888 (workflow_audit path anchoring, 0-AI-MANIFEST.deed), standards#1084 (harden-runner in standards-only workflows). Ledger: dev-notes/audits/hypatia-issue-sweep/2026-09-30.md.