feat(settings): add estate-wide settings drift detector

[hyperpolymath]

Why this, and not a re-apply of the canon

Repository settings are not in git. Nothing reviews them, nothing diffs them, and a change leaves no artefact — no commit, no PR, no run.

The estate measures 97–100% compliant with config/settings/repo.json at any given moment, so re-applying the canon gains almost nothing. What was missing is anything that notices a setting moving.

Two measurements from 2026-09-14 motivate it:

1. The allowlist moved under an active investigation. 239 repositories went from allowed_actions: selected with an empty patterns_allowed and verified_allowed: false, to a 92-pattern allowlist with verified_allowed: true — between a census and the write meant to fix it. Nobody could say when, by whom, or whether it was deliberate, because no instrument was watching.
2. Zero of the 301 repos calling mirror.yml had all seven *_MIRROR_ENABLED variables, and roughly a third had none. Each forge job is gated on if: vars.<FORGE>_MIRROR_ENABLED == 'true', so those runs reported green while backing up nowhere. A skip is not a red.

Why the first one matters more than it looks: an empty patterns_allowed refuses every third-party action, including transitively through a reusable workflow, because actions are judged against the caller repo's allow-list. Runs then die at startup_failure with jobs.total_count == 0 and emit no check run — a required context never reports and the repo looks greener than a healthy one.

What it does

scripts/check-settings-drift.sh compares live settings against the canon and emits TSV (repo, key, expected, actual); exit 0 clean, 1 drift, 2 usage/environment. It checks the repo block (skipping per_repo_deltas_allowed, and security_and_analysis via the per-repo GET, skipped on private per the canon's own repo_private_overrides), the Actions permissions block, workflow permissions, and — only where .github/workflows/mirror.yml exists — the seven forge variables.

.github/workflows/settings-drift-detect.yml runs it weekly and on dispatch, uploads the TSV, and opens or updates a single tracking issue. It reports only — it never mutates a setting or another repository, consistent with the estate guardrail that unattended cross-repo mutation is a human decision. Settings are exactly where that guardrail matters most.

Two traps handled explicitly

• The subshell trap. The scan loop uses process substitution, not gh repo list | while read. A pipe runs the loop body in a subshell, so every drift flag set inside is discarded when the subshell exits and the script always reports success — a drift detector structurally incapable of reporting drift. Found before shipping and proved with a control: process-substitution 1, pipe 0.
• The false-clean trap. actions/permissions is admin-scoped and security_and_analysis is absent from LIST endpoints entirely, so a token without admin scope produces a clean result it has no right to. The workflow probes that read first and fails with exit 2 rather than reporting an empty sweep — otherwise the guard would answer a different question than its consumer.

Verification

• Both uses: refs are present verbatim in .github/workflows/actions.lock. The lockfile matches literal ref strings rather than resolved commits, so an unlisted SHA startup-kills the run; my first draft pinned an upload-artifact SHA the lockfile does not carry, and that was caught before pushing.
• bash -n clean; workflow parses.
• Positive control, 6-repo slice: 31 drift rows, exit 1 — including proven-tests-and-benches missing all seven mirror variables, which is the precise fault class this was written for.

Calibration — what a hit does and does not prove

A row means live differs from canon. It does not by itself mean anything is broken: the canon documents per_repo_deltas_allowed, and private repos legitimately differ. Equally, a clean result is not proof CI is healthy — a startup failure can be event-specific and invisible in every setting and every file (measured the same day: the same commit that died with jobs=0 on push started 7 jobs on workflow_dispatch). The preamble in the script states this at length so a future reader does not over-read a clean sweep.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QMTyDv9CoJo5PfeNzyp519

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 3e670e01-0354-4515-b01b-b64845b981f1

📥 Commits

Reviewing files that changed from the base of the PR and between abd9757 and 5fe2cca.

📒 Files selected for processing (2)

• .github/workflows/settings-drift-detect.yml
• scripts/check-settings-drift.sh

---

📝 Summary

Summary by CodeRabbit

• New Features
  • Added automated detection of repository settings that differ from the canonical configuration.
  • Scheduled weekly scans, with optional manual runs and repository scan limits.
  • Generates a downloadable drift report and creates or updates a tracking issue when differences are found.
  • Reports action permission and mirror-forge configuration mismatches without changing repository settings.

Walkthrough

Adds a report-only settings drift scanner and a scheduled workflow. The scanner compares repositories with config/settings/repo.json. The workflow uploads drift results and creates or updates a tracking issue when drift is detected.

Changes

Settings drift detection

Layer / File(s)	Summary
Scan contract and repository iteration scripts/check-settings-drift.sh	Defines scan options, exit codes, canonical settings loading, TSV output, and repository enumeration.
Repository settings checks scripts/check-settings-drift.sh	Checks repository settings, security status, Actions permissions, selected action allowlists, workflow permissions, and mirror variables.
Scheduled scan and issue reporting .github/workflows/settings-drift-detect.yml	Runs the scan on a schedule or manual dispatch, verifies token access, uploads drift.tsv, and creates or updates a Settings drift issue.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Workflow
  participant Scanner
  participant GitHubAPI
  participant IssueTracker
  Workflow->>Scanner: Run check-settings-drift.sh
  Scanner->>GitHubAPI: Read repository settings
  GitHubAPI-->>Scanner: Return live settings
  Scanner-->>Workflow: Return drift.tsv and exit code
  Workflow->>IssueTracker: Create issue or add comment when drift is found

Loading

Suggested reviewers: joshuajewell

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{A rabbit checks each setting bright
Against the canon day and night
A tidy report hops into view
An issue records what drift can do
Carrots celebrate the queue}

_{Comment @coderabbitai help to get the list of available commands.}

[sonarqubecloud]

Quality Gate passed

Issues
16 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud