diff --git a/.changeset/6906-hook-selftests-collector.md b/.changeset/6906-hook-selftests-collector.md new file mode 100644 index 0000000000..a54a1a44ab --- /dev/null +++ b/.changeset/6906-hook-selftests-collector.md @@ -0,0 +1,31 @@ +--- +--- + +CI-only change: no published package's `src/` changed. + +`.github/workflows/hook-selftests.yml` now **discovers** the matrices it runs +instead of hand-enumerating them. Its single step runs +`find .claude/hooks -type f -name '*.selftest.sh' | sort` and executes each hit, +so a new `.claude/hooks/**/*.selftest.sh` is picked up with no edit to the +workflow — the property the file's own header already claimed ("the standing +caller for `.claude/hooks/*.selftest.sh`", whose matrices "are meant to grow in +`.claude/**` WITHOUT this file being touched") and did not have: the previous +`steps:` were a hand-kept list, and `guard-tree-enum.selftest.sh` had to be +added to it by hand while the sibling repo's collector picked the same file up +automatically. + +Ported from objectstack's collector in its `lint.yml`, keeping both load-bearing +properties: an **empty** discovery is RED (a step that verified nothing is not a +pass, so a renamed or moved directory cannot degrade the gate into a silent +no-op), and the loop **tolerates and collects** rather than sequencing, so one +red matrix cannot abort the run and hide the others — it still fails the job and +names every matrix that failed. + +Coverage is unchanged, measured rather than asserted: discovery returns exactly +the three self-tests the removed steps ran, and no self-test file was moved, +renamed or skipped. Does not modify the hooks or their self-tests (`.claude/**` +is governed surface). Paired write: +`content/docs/guide/ci-cd-pipeline.md` (the workflow's own section hand-listed +two of the three matrices). `scripts/dependabot-merge-gate.mjs` was read and +left untouched — its `OPTIONAL_CONTEXTS` description of this check is already +count-free. diff --git a/.github/workflows/hook-selftests.yml b/.github/workflows/hook-selftests.yml index 61dae6340d..63e3ed807e 100644 --- a/.github/workflows/hook-selftests.yml +++ b/.github/workflows/hook-selftests.yml @@ -31,12 +31,54 @@ name: Hook Self-Tests # reads. If a self-test itself is ever found broken, that is a finding for a # *separate* PR against the governed surface, not something to patch here. # +# ── ⭐ DISCOVERED at run time, never listed (objectui#6906) ────────────────── +# +# The single `run:` step below finds its matrices with +# `find .claude/hooks -type f -name '*.selftest.sh'`. It used to be a hand-kept +# ENUMERATION of one `run:` step per matrix, and that shape was this workflow's +# own defect one level up: the name at the top of this file claims to be the +# standing caller for all of them, and nothing made the claim true. A new matrix +# shipped UNCALLED until someone remembered to edit this file — measured, not +# argued: `guard-tree-enum.selftest.sh` landed in both repos on the same card and +# objectstack's collector picked it up with no workflow edit, while here it had +# to be added by hand. +# +# That defect is structural rather than carelessness, and the header below +# already named the cause: this file is a RUNNER, so the matrices it calls are +# meant to grow in `.claude/**` WITHOUT this file being touched (both #6042 / +# PR #6087 and #5789 / PR #6046 do exactly that, and are right to). Discovery is +# what finally makes that independence hold for a whole new matrix FILE and not +# only for cases inside an existing one. It also unblocks #6451 — +# `guard-main-checkout.sh` has no self-test at all, and whoever writes that +# matrix would otherwise have to remember this file or watch their work never +# run. +# +# `find`, not a flat glob, so a matrix in a subdirectory is not a silent miss +# either. Ported from objectstack's collector in its `lint.yml` +# ("Claude hook guard self-tests"), including both of the load-bearing +# properties below — a port that drops either is worse than no change. +# +# ⭐ 1. An EMPTY discovery is RED, not green. A renamed, moved or deleted +# directory would otherwise make this step pass by running nothing, +# which is the identical green line this whole workflow exists to +# distrust: a step that verified nothing is not a pass. The count is +# printed on success too, so a SHRINKING set is visible in the log +# rather than inferred. +# +# ⭐ 2. Collected, not sequenced. A `run:` block executes under `bash -e`, so +# a bare `for … do "$selftest"; done` would abort at the FIRST red +# matrix and leave the rest unrun — neither green nor red, and nothing +# in the log tells those apart. The matrices are independent by +# construction (each builds its own throwaway fixture), so collecting +# loses nothing and the step still FAILS when any of them does, naming +# every one that failed. ⛔ Do not "simplify" it into a bare loop. +# # ── Why path-filtered, unlike `control-bytes.yml` ─────────────────────────── # # `control-bytes.yml`'s header explains at length why ITS gate carries no `paths` # filter: a markdown-only PR can trip a raw control byte just as easily as a # TypeScript one, so an unfiltered gate is the only kind that can see it. That -# reasoning does not transfer here — both self-tests build their own hermetic +# reasoning does not transfer here — the self-tests build their own hermetic # git fixtures and assert the CURRENT hook script's behaviour; nothing about a # docs-only or dependency-bump PR can move that needle. So this workflow mirrors # `changeset-guard.yml`'s shape instead: it fires only on a PR (or push to `main`) @@ -52,23 +94,26 @@ name: Hook Self-Tests # # Those counts stay, and the distinction is deliberate (objectui#6089): a # measurement is HISTORY. It is read together with its date and its sha, and a -# matrix that grows afterwards does not make it false — it makes it older. A -# step NAME is a LIVE CLAIM about the run it labels, so it carries no count at -# all. See the next section. +# matrix that grows afterwards does not make it false — it makes it older. Both +# have grown since (121 and 41 as of 2026-08-31, plus a third matrix), which is +# exactly what a dated measurement is allowed to do. A step NAME, by contrast, +# is a LIVE CLAIM about the run it labels, so it carries no count at all. See +# the next section. # -# ── Why the step names carry no case counts ────────────────────────────────── +# ── Why the step name carries no case count ────────────────────────────────── # -# Each `run:` step below is named for the self-test it calls and nothing else. -# Do not add "(N cases)" back to either name — this file used to do exactly -# that, and it was the declared≠actual shape: +# The `run:` step below is named for what it does and nothing else. Do not add +# "(N cases)" or "(N matrices)" to it — this file used to name one step per +# matrix WITH its case count, and it was the declared≠actual shape: # # - Nothing derived those numbers and nothing re-checked them. A step name # has no runtime behaviour, so a wrong count never turns a check red; it # just shows a wrong number on the checks page, which is the one place a # reviewer actually looks. # - The number is already published by the only thing that cannot get it -# wrong: each run's own tail prints `N passed, N failed`, produced by the -# matrix being counted. +# wrong: the step prints how many matrices it discovered, and each run's +# own tail prints `N passed, N failed`, produced by the matrix being +# counted. # - Where a count belongs beside prose, the hook header owns it WITH the # recipe to re-derive it: `guard-shared-stash.sh` states its total, how to # re-derive it (`grep -c '^expect ' ` plus the inline specials) @@ -76,18 +121,18 @@ name: Hook Self-Tests # describes its matrix with no count at all. A second copy here would be a # second thing to keep in sync, and the one with no recipe behind it. # -# The drift was structural, not carelessness: this file is a RUNNER, so the -# matrices it calls are meant to grow in `.claude/**` WITHOUT this file being -# touched (both #6042 / PR #6087 and #5789 / PR #6046 do exactly that, and are -# right to). Dropping the duplicate is what makes that independence safe. +# The same reasoning is why `scripts/dependabot-merge-gate.mjs`'s +# `OPTIONAL_CONTEXTS` description of this check is count-free, and must stay so. # # ── Fail-closed, deliberately ──────────────────────────────────────────────── # # Each self-test script already exits non-zero on any failing case (its own -# `pass`/`fail` tally), so a plain `run:` step is enough to fail the job — and -# therefore the workflow — the moment either matrix regresses. A gate that ran -# these and reported without failing the build would reproduce exactly the defect -# this workflow exists to close. +# `pass`/`fail` tally), and the collector below propagates that into the step's +# own exit status, so the job — and therefore the workflow — goes red the moment +# any matrix regresses. A gate that ran these and reported without failing the +# build would reproduce exactly the defect this workflow exists to close. +# ⛔ Never let the collector swallow an exit code: a green step over a red +# self-test looks identical to success. on: pull_request: @@ -119,27 +164,43 @@ jobs: - name: Checkout code uses: actions/checkout@v7 - # Hermetic: builds its own throwaway git repo, worktree and non-repo - # directory under $TMPDIR. Exits non-zero on any failing case. - - name: guard-main-checkout-bash self-test - run: .claude/hooks/guard-main-checkout-bash.selftest.sh - - # Hermetic in the same sense. Exits non-zero on any failing case. - - name: guard-shared-stash self-test - run: .claude/hooks/guard-shared-stash.selftest.sh - - # Hermetic in the same sense — it feeds the hook JSON payloads and needs - # no git fixture at all. Exits non-zero on any failing case. - # - # ⚠️ This step had to be ADDED BY HAND, and that is the standing cost of - # the shape above: the steps here are a hand-kept ENUMERATION of - # `.claude/hooks/*.selftest.sh`, so a new matrix ships uncalled unless - # someone remembers this file — the workflow name claims to be the - # standing caller for all of them, and nothing makes the claim true. - # objectstack's twin does not have this failure mode: its collector runs - # `find .claude/hooks -type f -name '*.selftest.sh'`, so a new self-test is - # picked up with no workflow edit. Converting this job to that shape is - # worth doing and is deliberately NOT done here (it is a change to the - # runner's design, not to the matrix it runs); it is filed separately. - - name: guard-tree-enum self-test - run: .claude/hooks/guard-tree-enum.selftest.sh + # Every matrix under `.claude/hooks/` is hermetic: it builds its own + # throwaway git repo, worktree and non-repo directory under $TMPDIR, and + # exits non-zero on any failing case. Discovery + the two load-bearing + # properties are argued in the header sections above — read them before + # editing this block. + - name: Claude hook guard self-tests + run: | + mapfile -t selftests < <(find .claude/hooks -type f -name '*.selftest.sh' | sort) + if [ "${#selftests[@]}" -eq 0 ]; then + echo "Hook self-tests — DISCOVERED NOTHING under .claude/hooks/." + echo "This step verified nothing, which is a failure and not a pass:" + echo "the matrices were moved, renamed or deleted. Re-point the search" + echo "above rather than deleting the step." + exit 1 + fi + echo "discovered ${#selftests[@]} hook self-test(s):" + printf ' %s\n' "${selftests[@]}" + echo "" + failed="" + run_self_test() { + echo "-- $*" + if "$@"; then + echo "PASS $*" + else + echo "FAIL $*" + failed="${failed} $*"$'\n' + fi + return 0 + } + for selftest in "${selftests[@]}"; do + run_self_test "$selftest" + done + if [ -n "$failed" ]; then + echo "" + echo "Hook self-tests — the following FAILED:" + printf "%s" "$failed" + exit 1 + fi + echo "" + echo "Hook self-tests — all ${#selftests[@]} ran and passed" diff --git a/content/docs/guide/ci-cd-pipeline.md b/content/docs/guide/ci-cd-pipeline.md index e628ffdae3..73496e924d 100644 --- a/content/docs/guide/ci-cd-pipeline.md +++ b/content/docs/guide/ci-cd-pipeline.md @@ -1681,11 +1681,23 @@ The rendered summary says that surface is **UNREAD**, never that it is clean **Trigger:** PR to `main`/`develop`, and push to `main`, **when `.claude/hooks/**` or this workflow file changes**. **Blocks a PR:** yes. -Runs `.claude/hooks/guard-main-checkout-bash.selftest.sh` and -`.claude/hooks/guard-shared-stash.selftest.sh` — the hermetic self-test matrices for -the two PreToolUse guards behind the rule both `CLAUDE.md` files state as binding: worktree-first, -and never `git stash` (AGENTS.md §9). Each self-test builds its own throwaway git fixture and -needs only `jq` and `git`, both preinstalled on `ubuntu-latest` — no install, no build. +Runs the hermetic self-test matrices for the PreToolUse guards behind the rules both +`CLAUDE.md` files state as binding: worktree-first, and never `git stash` (AGENTS.md §9). Each +self-test builds its own throwaway git fixture and needs only `jq` and `git`, both preinstalled +on `ubuntu-latest` — no install, no build. + +**Discovered at run time, never enumerated** +([#6906](https://github.com/objectstack-ai/objectui/issues/6906)). The single step runs +`find .claude/hooks -type f -name '*.selftest.sh'`, so a new matrix under `.claude/hooks/` is +picked up with **no edit to this workflow** — including one in a subdirectory, which a flat glob +would miss. It used to be one hand-written `run:` step per matrix, and that was this workflow's +own defect one level up: it names itself the standing caller for all of them while a new matrix +shipped uncalled until someone remembered to edit it. Two properties keep discovery honest, and +both are load-bearing: an **empty** discovery is **red** (a step that verified nothing is not a +pass, so a renamed or moved directory cannot degrade the gate into a silent no-op), and the loop +**tolerates and collects** rather than sequencing — a `run:` block executes under `bash -e`, so a +bare loop would abort at the first red matrix and leave the rest unrun, neither green nor red. +The step still fails when any matrix does, and names every one that failed. Before this workflow ([#5754](https://github.com/objectstack-ai/objectui/issues/5754)), nothing ran either matrix automatically: a hook is not imported by any package, so no unit test, type @@ -1699,11 +1711,11 @@ without a caller. **This job is a runner, not a rewrite.** It does not modify the hooks or their self-tests — `.claude/**` is governed surface. It only gives the existing matrices a caller that fails the -build the moment either one goes red, the same way any other required check does. +build the moment any one of them goes red, the same way any other required check does. **Path-filtered, unlike `control-bytes.yml`.** That gate carries no `paths` filter because a raw control byte can land in a markdown-only PR just as easily as a TypeScript one. That reasoning -does not transfer here: both self-tests assert the CURRENT hook script's behaviour against a +does not transfer here: the self-tests assert the CURRENT hook script's behaviour against a fixture they build themselves, so nothing about a docs-only or dependency-bump PR can move the result. This workflow instead mirrors `changeset-guard.yml`'s inverse-filter shape, firing only on a PR that touches `.claude/hooks/**` — which is also why `scripts/dependabot-merge-gate.mjs`