From f4ce2dfc6b705ef68168dc16471ccfda7f6d61f7 Mon Sep 17 00:00:00 2001 From: letur Date: Sun, 13 Sep 2026 08:36:45 +0200 Subject: [PATCH 1/3] fix(gate): read no operator state in check.sh or the selftests One commit now gets one ./scripts/check.sh verdict on CI, in a worker worktree and in the control-plane checkout. - check.sh queue no longer runs `queue.sh check` over the checkout's live records; `fleet-status.sh --records` validates them (and the registry map) instead, through the same validators. - check.sh automerge judges a fresh-clone root holding only the tracked example, rather than answering skip beside an operator auto-merge.conf. - check.sh yaml no longer validates the gitignored registry map; the onboarding selftest holds a generated map to that shape instead. - check.sh voice renders off the tracked glyph and agent defaults (install-extension.sh now honours FLEET_GLYPH_ROOT). - scripts/lib/selftest-env.sh pins git config, HOME, forge credentials, THURBOX_SESSION and every FLEET_* settings root for all seven selftests. - scripts/isolation-selftest.sh (check.sh isolation, CI job) re-runs the checks that could leak in a poisoned copy of the tree under a hostile host. Claude-Session: https://claude.ai/code/session_01TVf5c2rWYcoVREhFKgquNJ --- .agents/skills/fleet-queue/SKILL.md | 10 +- .agents/skills/update-fleet/SKILL.md | 6 + .github/workflows/ci.yml | 13 ++ .pre-commit-config.yaml | 2 +- AGENTS.md | 7 + scripts/check.sh | 115 ++++++---- scripts/fleet-status-selftest.sh | 64 +++++- scripts/fleet-status.sh | 11 +- scripts/install-extension.sh | 7 +- scripts/install-selftest.sh | 13 +- scripts/isolation-selftest.sh | 317 +++++++++++++++++++++++++++ scripts/lib/check_yaml.py | 90 +++++--- scripts/lib/fleet_status.py | 85 +++++++ scripts/lib/queue.py | 27 ++- scripts/lib/selftest-env.sh | 75 +++++++ scripts/onboarding-selftest.sh | 50 +++-- scripts/pane-selftest.sh | 8 + scripts/queue-selftest.sh | 13 +- scripts/queue.sh | 2 +- scripts/reconcile-selftest.sh | 5 + scripts/sync-selftest.sh | 6 + 21 files changed, 794 insertions(+), 132 deletions(-) create mode 100755 scripts/isolation-selftest.sh create mode 100644 scripts/lib/selftest-env.sh diff --git a/.agents/skills/fleet-queue/SKILL.md b/.agents/skills/fleet-queue/SKILL.md index b2cda3c..3256e99 100644 --- a/.agents/skills/fleet-queue/SKILL.md +++ b/.agents/skills/fleet-queue/SKILL.md @@ -957,10 +957,12 @@ it existed. So **the repo does not back your queue up.** Say that plainly when someone assumes otherwise; `.gitignore`'s header owns the reasoning. -`./scripts/check.sh queue` validates your records and re-proves the ordering and -wake claims against a throwaway queue. It runs in the gate, so a change that -quietly makes the queue serialize by default fails there rather than in a run -six weeks later. +`./scripts/check.sh queue` re-proves the ordering and wake claims against a +throwaway queue. It runs in the gate, so a change that quietly makes the queue +serialize by default fails there rather than in a run six weeks later. It does +**not** read your records — the gate reads no operator state, so one commit gets +one verdict in every checkout. `./scripts/fleet-status.sh --records` validates +them, and `./scripts/queue.sh check` lists every problem. ## The loop diff --git a/.agents/skills/update-fleet/SKILL.md b/.agents/skills/update-fleet/SKILL.md index 6c6f38e..4600090 100644 --- a/.agents/skills/update-fleet/SKILL.md +++ b/.agents/skills/update-fleet/SKILL.md @@ -244,6 +244,12 @@ and this skill changes no tracked state. Print what failed and hand it over. A green gate is worth one line. A red one is worth the failing check's output. +**The gate says nothing about the operator's own records** — it reads no queue +record and no registry map, so its verdict here is the verdict CI gave. Whether +those are sound after the sync is `./scripts/fleet-status.sh --records`: report +its two lines, and a problem there as one in the records, not in what +arrived. + ## 8. Hand over the stale lead — the step that cannot be automated If an `INSTRUCTION_PATH` moved, the running Mission Control session **froze diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f9424e1..a975969 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -213,6 +213,18 @@ jobs: # install.sh and pane-ask.sh, driven end to end; same stubs, same job. - run: ./scripts/check.sh onboarding install + # Unfiltered, because what it guards is every check's INPUTS: that no check + # or selftest reads a queue record, a registry map, a gitignored setting, the + # caller's HOME or their git config. A change to any script can break that, + # and a runner holds none of those things, so only a job that builds them — + # a poisoned checkout under a hostile host — can see it. Seconds. + isolation: + name: Isolation + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7 + - run: ./scripts/check.sh isolation + # Single required status check. Every job feeds this gate; a branch ruleset # requires only "All Checks", so adding or removing jobs never needs a # ruleset change — but a new job MUST be added to `needs:` below, or it can @@ -231,6 +243,7 @@ jobs: - skills - pane - onboarding + - isolation runs-on: ubuntu-latest steps: - name: Verify all checks passed diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index a5623b1..728a8c6 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -27,7 +27,7 @@ repos: pass_filenames: false - id: yaml - name: YAML parses, registry shape holds + name: YAML parses entry: ./scripts/check.sh yaml language: system files: '\.ya?ml$' diff --git a/AGENTS.md b/AGENTS.md index 792e20c..eb6bbe4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -285,6 +285,13 @@ runs it, the prek hooks run it, and `.no-mistakes.yaml` points its `lint` command at it, so a green local run and a green pull request mean the same thing. +**The gate reads no operator state** — not the queue's records, the registry +map, a gitignored `*.conf`, your HOME or your git config — so one commit gets +one verdict in a worker's worktree, on CI and in this checkout alike. +`./scripts/fleet-status.sh --records` is where your live records are validated +now; `check.sh isolation` is what keeps the gate from reading them +again, and every selftest goes through `scripts/lib/selftest-env.sh`. + Changes that open a pull request land by **squash merge** — the only merge method the remote allows — so the pull request title becomes the commit on `main`. `CONTRIBUTING.md` owns that process. diff --git a/scripts/check.sh b/scripts/check.sh index 3cc158d..08581b7 100755 --- a/scripts/check.sh +++ b/scripts/check.sh @@ -13,8 +13,20 @@ # scripts/check.sh --fix markdown # apply the fixes a check can apply # # Checks: shell, markdown, yaml, profiles, queue, reconcile, status, skills, -# pane, voice, automerge, onboarding, install, sync. Only `markdown` has a fixer; `--fix` is a no-op for -# the rest, so `scripts/check.sh --fix` is always safe to run. +# pane, voice, automerge, onboarding, install, sync, isolation. Only `markdown` +# has a fixer; `--fix` is a no-op for the rest, so `scripts/check.sh --fix` is +# always safe to run. +# +# IT READS NO OPERATOR STATE. One commit gets one verdict — on CI, in a worker's +# worktree, and in the control-plane checkout — so no check reads what a running +# fleet wrote into a checkout: the queue's records, the registry map, the +# gitignored `orchestration/*.conf`, the reconciler's runtime, or the caller's +# HOME and git config. Checks read tracked files; every selftest runs through +# `scripts/lib/selftest-env.sh`. The live half moved and did not vanish: +# `./scripts/fleet-status.sh --records` validates the operator's queue records +# and registry map, and `./scripts/queue.sh check` still validates the queue on +# its own. `isolation` holds the line, by re-running the +# checks that could leak in a poisoned copy of the tree under a hostile host. # # Requires: shellcheck, rumdl, python3 (with PyYAML), lua. A missing tool # fails the check rather than skipping it — a gate that silently passes when @@ -97,7 +109,7 @@ check_yaml() { fi if python3 scripts/lib/check_yaml.py "${files[@]}"; then - ok "yaml: ${#files[@]} tracked files parse, registry shape holds" + ok "yaml: ${#files[@]} tracked files parse" else fail "yaml" fi @@ -125,26 +137,22 @@ check_profiles() { fi } -# The task queue, in two halves. `queue.sh check` validates the local -# records — a blocker naming a task that no longer exists, a state word nobody -# defined — and says so and passes when the queue has never been used, the way -# check_yaml.py treats an unsynced registry. +# The task queue's claims. The queue makes claims that are easy to invert by +# accident: that independent work goes out all at once, that file overlap does +# NOT serialize, that a turn ending is not a task finishing. Each is a test +# against a throwaway queue, so a change that quietly reverses one fails here +# rather than in a run six weeks later. # -# `queue-selftest.sh` is the other half and the more important one. The queue -# makes claims that are easy to invert by accident: that independent work goes -# out all at once, that file overlap does NOT serialize, that a turn ending is -# not a task finishing. Each is a test against a throwaway queue, so a change -# that quietly reverses one fails here rather than in a run six weeks later. +# NOT this checkout's own records. `queue.sh check` over them used to run here, +# which validated 72 live topics in the control-plane checkout and an empty +# directory everywhere else — so a record nobody touched could fail a commit +# there that CI passed. Those records are the operator's, and +# `fleet-status.sh --records` is where they are validated now. check_queue() { need python3 queue || return - local out - if ! out="$(./scripts/queue.sh check)"; then - fail "queue: ./scripts/queue.sh check" - return - fi if ./scripts/queue-selftest.sh >/dev/null; then - ok "queue: ${out#queue check: }, ordering and wake claims hold" + ok "queue: ordering and wake claims hold" else # Re-run visibly: a failing claim is the whole message. ./scripts/queue-selftest.sh @@ -547,9 +555,15 @@ check_voice() { } printf 'OPERATOR_NAME=GATEOP\nASSISTANT_NAME=GATEAI\n' >"$tmp/voice.conf" + # The glyph and the agent render into the same manifest, and off the + # tracked defaults only: an operator's session-glyphs.conf or agent.conf in + # this checkout decides nothing about this commit. + mkdir -p "$tmp/defaults/orchestration" + cp orchestration/*.example.conf "$tmp/defaults/orchestration/" + local out="$tmp/FLEET.rendered.md" report - if ! report="$(FLEET_VOICE_CONF="$tmp/voice.conf" \ - ./scripts/install-extension.sh --render-only "$tmp" 2>&1)"; then + if ! report="$(FLEET_VOICE_CONF="$tmp/voice.conf" FLEET_GLYPH_ROOT="$tmp/defaults" \ + FLEET_AGENT_ROOT="$tmp/defaults" ./scripts/install-extension.sh --render-only "$tmp" 2>&1)"; then fail "voice: install-extension.sh --render-only failed: $report" miss=1 elif [ ! -f "$out" ]; then @@ -597,24 +611,30 @@ check_automerge() { miss=1 fi - # And the set itself, read the way `shepherd` reads it. A checkout with an - # operator's own auto-merge.conf in it answers about that file instead, so - # the gate reads the tracked one directly there rather than passing on a - # result about somebody's private list. - local shipped - if [ -f orchestration/auto-merge.conf ]; then - shipped="skip" - else - shipped="$(FLEET_AUTO_MERGE_REPOS='' python3 -c ' + # And the set itself, read the way `shepherd` reads it — from a root that + # holds the tracked example and nothing else, which is exactly what a fresh + # clone holds. Never this checkout's root: an operator's auto-merge.conf + # there would answer instead, and the gate used to answer `skip` rather than + # judge somebody's private list, so the claim went unproven in the one + # checkout that has one. + local shipped fresh + fresh="$(mktemp -d)" || { + fail "automerge: could not make a temp directory for the fresh-clone reading" + return + } + mkdir -p "$fresh/orchestration" + cp "$example" "$fresh/orchestration/" + shipped="$(env -u FLEET_AUTO_MERGE_ROOT FLEET_AUTO_MERGE_REPOS='' python3 - "$fresh" <<'PY' 2>&1 import sys sys.path.insert(0, "scripts/lib") import queue as q -print("entries=" + (" ".join(sorted(q.auto_merge_repos(q.checkout_root()))) or "none")) -' 2>&1)" - if [ "$shipped" != "entries=none" ]; then - fail "automerge: a fresh clone would inherit a merge allowlist: $shipped" - miss=1 - fi +print("entries=" + (" ".join(sorted(q.auto_merge_repos(sys.argv[1]))) or "none")) +PY +)" + rm -rf "$fresh" + if [ "$shipped" != "entries=none" ]; then + fail "automerge: a fresh clone would inherit a merge allowlist: $shipped" + miss=1 fi # THE SAME RULE FOR EVERY TRACKED SETTING. An owner, a repository, a @@ -665,7 +685,7 @@ print(" ".join(sorted(q.PUBLISH_METHODS))) { fail "automerge: the publish methods are '''$shapes''', and must be artifact shapes"; miss=1; } [ "$miss" -eq 0 ] && - ok "automerge: no tracked setting names a repository, a tool or an agent" + ok "automerge: no tracked setting names a repository, a tool or an agent, and a fresh clone merges nowhere" } # THE SETUP NOBODY RE-RUNS. Onboarding's scripts — preflight, discover-owners, @@ -710,6 +730,24 @@ check_install() { fi } +# ONE COMMIT, ONE VERDICT. The header's promise that no check reads operator +# state, held by building the worst case: a poisoned copy of the tree under a +# hostile host, with every check that reads settings or records and every +# selftest cheap enough to run twice re-run inside it. Seconds, because the two +# slow selftests are held to the shared helper rather than re-run. +check_isolation() { + need git isolation || return + need jq isolation || return + need python3 isolation || return + + if ./scripts/isolation-selftest.sh >/dev/null 2>&1; then + ok "isolation: a poisoned checkout under a hostile host gets the same verdict" + else + ./scripts/isolation-selftest.sh + fail "isolation: scripts/isolation-selftest.sh" + fi +} + checks=() for arg in "$@"; do case "$arg" in @@ -719,7 +757,7 @@ for arg in "$@"; do done if [ ${#checks[@]} -eq 0 ]; then - checks=(shell markdown yaml profiles queue reconcile status skills pane voice automerge onboarding install sync) + checks=(shell markdown yaml profiles queue reconcile status skills pane voice automerge onboarding install sync isolation) fi for c in "${checks[@]}"; do @@ -738,8 +776,9 @@ for c in "${checks[@]}"; do automerge) check_automerge ;; onboarding) check_onboarding ;; install) check_install ;; + isolation) check_isolation ;; *) - printf 'error: unknown check %q (want: shell markdown yaml profiles queue reconcile status skills pane voice automerge onboarding install sync)\n' "$c" >&2 + printf 'error: unknown check %q (want: shell markdown yaml profiles queue reconcile status skills pane voice automerge onboarding install sync isolation)\n' "$c" >&2 exit 2 ;; esac diff --git a/scripts/fleet-status-selftest.sh b/scripts/fleet-status-selftest.sh index 0d7d7a8..24f71b3 100755 --- a/scripts/fleet-status-selftest.sh +++ b/scripts/fleet-status-selftest.sh @@ -72,6 +72,11 @@ for tool in python3 git; do done tmp="$(mktemp -d)" +# Nothing of this machine's or this checkout's operator reaches this run; +# scripts/lib/selftest-env.sh says what that covers. +# shellcheck source=scripts/lib/selftest-env.sh +. scripts/lib/selftest-env.sh +selftest_isolate "$tmp/env" export FLEET_QUEUE_DIR="$tmp/queue" # `topic add` also opens a run log; keep this run's out of the operator's # orchestration/runs/, the same way FLEET_QUEUE_DIR keeps its queue out. @@ -100,7 +105,7 @@ stubbed="$(sandbox "$tmp/bin-stubbed" "${BASE_TOOLS[@]}")" # --- a queue with something in it ------------------------------------------- -"$QUEUE" topic add selftest --title "Selftest topic" --prompt 'the prompt, verbatim' >/dev/null 2>&1 +PATH="$bare" "$QUEUE" topic add selftest --title "Selftest topic" --prompt 'the prompt, verbatim' >/dev/null 2>&1 cat >"$tmp/brief.md" <<'MD' ## What to do @@ -118,15 +123,15 @@ None. It is done. MD -"$QUEUE" add selftest dispatched-task --title "A dispatched task" --repo "$tmp/repo" \ +PATH="$bare" "$QUEUE" add selftest dispatched-task --title "A dispatched task" --repo "$tmp/repo" \ --branch t/dispatched --touches FLEET.md --brief-file "$tmp/brief.md" >/dev/null -"$QUEUE" add selftest ready-task --title "A ready task" --repo "$tmp/repo" \ +PATH="$bare" "$QUEUE" add selftest ready-task --title "A ready task" --repo "$tmp/repo" \ --branch t/ready --touches FLEET.md --brief-file "$tmp/brief.md" >/dev/null -"$QUEUE" add selftest waiting-task --title "A waiting task" --repo "$tmp/repo" \ +PATH="$bare" "$QUEUE" add selftest waiting-task --title "A waiting task" --repo "$tmp/repo" \ --branch t/waiting --brief-file "$tmp/brief.md" >/dev/null -"$QUEUE" block selftest/03-waiting-task --on selftest/01-dispatched-task \ +PATH="$bare" "$QUEUE" block selftest/03-waiting-task --on selftest/01-dispatched-task \ --kind semantic-dependency --why 'consumes the flag the first one adds' >/dev/null -"$QUEUE" attach selftest/01-dispatched-task 11111111-1111-1111-1111-111111111111 >/dev/null +PATH="$bare" "$QUEUE" attach selftest/01-dispatched-task 11111111-1111-1111-1111-111111111111 >/dev/null # --- 1. every probe missing, and it still answers ---------------------------- @@ -187,6 +192,53 @@ else fi expect "the checkout still reports with no queue at all" "CHECKOUT" "$gone" +# --- 3b. the operator's records are validated here, and not in the gate ------ +# +# `scripts/check.sh` reads no operator state, so one commit gets one verdict in +# every checkout — which makes `--records` the only place a malformed queue +# record or a registry map of the wrong shape is reported. It is a flag and not +# a section of the screen, because validating opens every record and the +# screen promises never to open an archived one. It is held to this file's +# first rule all the same: a broken record costs lines, never the exit code. + +refute "the screen validates nothing, so it opens no record it need not" "RECORDS" "$out" +ok_rec="$(PATH="$bare" "$STATUS" --records 2>&1)" +expect "a sound queue's records read as ok" "queue: ok — 1 topic(s), 3 task(s)" "$ok_rec" +expect "an unsynced registry map is not a problem" "registry: ok — not synced yet" "$ok_rec" + +broken="$tmp/broken-queue" +mkdir -p "$broken/bad/01-bad" +printf 'slug: bad\ntitle: A bad topic\n' >"$broken/bad/topic.yaml" +cat >"$broken/bad/01-bad/task.yaml" <<'YAML' +id: 01-bad +topic: bad +title: A bad record +state: half-done +repo: /nowhere +branch: bad/branch +blocked_by: + - {task: bad/42-gone, kind: semantic-dependency, why: made up for this test} +YAML +printf 'owners: not-a-list\n' >"$tmp/bad-registry.yaml" +rec="$(PATH="$bare" FLEET_QUEUE_DIR="$broken" FLEET_REGISTRY_FILE="$tmp/bad-registry.yaml" "$STATUS" --records 2>&1)" +rc=$? +if [ "$rc" -eq 0 ]; then + pass "broken records cost lines, never the command" +else + fail "broken records cost lines, never the command" "exit $rc${nl}$rec" +fi +expect "RECORDS names a state nobody defined" "state 'half-done' is not one of" "$rec" +expect "and a blocker naming a task that does not exist" "bad/42-gone, which does not exist" "$rec" +expect "and a record with no brief" "no BRIEF.md" "$rec" +expect "and says which command lists every one" "queue.sh check" "$rec" +expect "and a registry map of the wrong shape" "owners missing or not a list" "$rec" +recjs="$(PATH="$bare" FLEET_QUEUE_DIR="$broken" "$STATUS" --records --json 2>&1)" +expect "--records --json carries the same problems" "is not one of" "$recjs" + +printf 'owners:\n - name: octo\n repos: []\ntotals:\n repos: 0\n' >"$tmp/good-registry.yaml" +rec="$(PATH="$bare" FLEET_REGISTRY_FILE="$tmp/good-registry.yaml" "$STATUS" --records 2>&1)" +expect "a registry map of the right shape reads as ok" "registry: ok — 0 repos across 1 owners" "$rec" + # --- 4. the state vocabulary is not flattened -------------------------------- # # THE TEST THIS FILE EXISTS FOR. `uncovered` means "wired to report nothing", diff --git a/scripts/fleet-status.sh b/scripts/fleet-status.sh index 263c555..e6b0524 100755 --- a/scripts/fleet-status.sh +++ b/scripts/fleet-status.sh @@ -11,6 +11,7 @@ # scripts/fleet-status.sh # the screen # scripts/fleet-status.sh --json # the same reading, machine-readable # scripts/fleet-status.sh --fuel # the fuel section alone, one field per line +# scripts/fleet-status.sh --records # validate your queue records and registry map # # IT DEGRADES AND NEVER FAILS. No thurbox, no `gh`, no network, no queue: # each costs exactly its own section, which then says what it could not @@ -37,8 +38,16 @@ # own fields under their own names, so the pane and this screen cannot come to # different conclusions about what quota-axi said. # +# `--records` IS THE OPERATOR'S HEALTH CHECK. Whether the live queue records and +# the registry map are sound is answered here, and not by `scripts/check.sh`, +# which reads no operator state so that one commit gets one verdict in every +# checkout. A problem there is about the records, never about the code — run +# it after a sync, or whenever a queue command reports something odd. It is a +# flag and not a section because it opens every record, archived topics' +# included, and the screen promises never to. +# # Environment: FLEET_QUEUE_DIR, honoured exactly as scripts/queue.sh honours -# it. +# it, and FLEET_REGISTRY_FILE, which relocates the registry map the same way. # # Requires: python3 (with PyYAML). thurbox-cli, gh, git and quota-axi are each # optional and cost only their own section — quota-axi in particular is a tool diff --git a/scripts/install-extension.sh b/scripts/install-extension.sh index 2397296..ae3ffde 100755 --- a/scripts/install-extension.sh +++ b/scripts/install-extension.sh @@ -153,8 +153,11 @@ esac # The glyph setting, from the operator's own copy when there is one and from the # tracked defaults when there is not. Read as DATA and never sourced: this file # is a setting, and a setting that can execute is a different kind of file. -GLYPH_CONF="$REPO_ROOT/orchestration/session-glyphs.conf" -[ -f "$GLYPH_CONF" ] || GLYPH_CONF="$REPO_ROOT/orchestration/session-glyphs.example.conf" +# FLEET_GLYPH_ROOT relocates it, as it does for queue.py's worker spawns, so the +# gate's render reads the tracked default and never an operator's override. +GLYPH_ROOT="${FLEET_GLYPH_ROOT:-$REPO_ROOT}" +GLYPH_CONF="$GLYPH_ROOT/orchestration/session-glyphs.conf" +[ -f "$GLYPH_CONF" ] || GLYPH_CONF="$GLYPH_ROOT/orchestration/session-glyphs.example.conf" [ -f "$GLYPH_CONF" ] || die "missing the glyph setting: $GLYPH_CONF" glyph_setting() { diff --git a/scripts/install-selftest.sh b/scripts/install-selftest.sh index d455982..49461d8 100755 --- a/scripts/install-selftest.sh +++ b/scripts/install-selftest.sh @@ -40,14 +40,10 @@ set -uo pipefail cd "$(dirname "${BASH_SOURCE[0]}")/.." || exit 2 REPO="$PWD" -# The caller's git and gh environment decide nothing here; see -# onboarding-selftest.sh, which argues each of these. -unset GIT_CONFIG_GLOBAL GIT_CONFIG_SYSTEM GH_TOKEN GITHUB_TOKEN GH_HOST GITLAB_HOST +# The caller's git and forge environment decide nothing here either: +# scripts/lib/selftest-env.sh clears them once the temp directory exists, and +# onboarding-selftest.sh argues each one. unset FLEET_DIR FLEET_REPO FLEET_BRANCH -for ((_i = 0; _i <= ${GIT_CONFIG_COUNT:-0}; _i++)); do - unset "GIT_CONFIG_KEY_$_i" "GIT_CONFIG_VALUE_$_i" -done -unset GIT_CONFIG_COUNT _i nl=$'\n' failed=0 @@ -88,6 +84,9 @@ tmp="$(mktemp -d)" # Resolved, so a path printed by a script compares equal to the one built here # on a machine whose temp directory is a symlink. tmp="$(cd "$tmp" && pwd -P)" +# shellcheck source=scripts/lib/selftest-env.sh +. scripts/lib/selftest-env.sh +selftest_isolate "$tmp/env" # --- the sandbox -------------------------------------------------------------- diff --git a/scripts/isolation-selftest.sh b/scripts/isolation-selftest.sh new file mode 100755 index 0000000..d9cf924 --- /dev/null +++ b/scripts/isolation-selftest.sh @@ -0,0 +1,317 @@ +#!/usr/bin/env bash +# Prove that the gate reads nothing a running fleet wrote. +# +# One commit must give the same `./scripts/check.sh` verdict wherever it runs: +# CI, a worker's worktree, and the operator's own control-plane checkout with +# its live queue, its gitignored settings and its owner's global git config. +# Only the last of those holds any of that, so it is the one place a leak shows +# — a record nobody touched turns a green commit red there, or a claim CI +# proved is quietly skipped there — and it is the one place nobody reviews a +# gate run. So this builds that worst case on purpose: +# +# A POISONED COPY of the tree under test: a malformed queue record and an +# OPERATOR.md, an auto-merge.conf naming a repository, publish, agent, glyph +# and voice settings with odd values, a rendered extension.toml, a reconciler +# runtime directory, and a registry map of the wrong shape. All of it is made +# up here. Nothing is ever copied from a real control plane. +# +# A HOSTILE HOST: a HOME, an XDG git config and a GIT_CONFIG_GLOBAL that sign +# every commit with a gpg that fails, route every hook to one that fails, and +# name `trunk` the default branch; a thurbox hosts.toml; forge credentials and +# a THURBOX_SESSION in the environment; and tripwire `gh`, `glab`, +# `thurbox-cli`, `quota-axi` and `ssh` first on PATH, each recording that it +# was run. +# +# Then, in that copy and under that host, it runs every CHECK that reads +# settings or records, and every SELFTEST cheap enough to run twice. Each must +# pass exactly as it does on a clean runner. The two expensive selftests — +# queue and reconcile — are not run a second time: §3 holds every selftest to +# the one shared helper, `scripts/lib/selftest-env.sh`, and §2 proves that +# helper against the same hostile host. +# +# Every assertion names the leak it guards, so a FAIL line is the diagnosis. +# +# Usage: scripts/isolation-selftest.sh (also: ./scripts/check.sh isolation) +# +# Requires: git, jq and python3 (with PyYAML) — the dependencies of the checks +# it re-runs. + +set -uo pipefail + +cd "$(dirname "${BASH_SOURCE[0]}")/.." || exit 2 + +nl=$'\n' +failed=0 +tmp="" +trap '[ -n "$tmp" ] && rm -rf "$tmp"' EXIT + +pass() { printf ' \033[32mok\033[0m %s\n' "$1"; } + +fail() { + printf ' \033[31mFAIL\033[0m %s\n' "$1" >&2 + if [ $# -gt 1 ]; then printf '%s\n' "$2" | sed 's/^/ /' >&2; fi + failed=1 +} + +expect() { + if grep -qF -- "$2" <<<"$3"; then pass "$1"; else fail "$1" "wanted: $2${nl}--- got ---${nl}$3"; fi +} + +refute() { + if grep -qF -- "$2" <<<"$3"; then fail "$1" "did not want: $2${nl}--- got ---${nl}$3"; else pass "$1"; fi +} + +# A check passed: exit 0. When it did not, its own FAIL lines are the message, +# and its last lines when it printed none. +expect_green() { + [ "$2" -eq 0 ] && { + pass "$1" + return + } + local why + why="$(grep -F -A6 FAIL <<<"$3" | head -60)" + fail "$1" "exit $2${nl}${why:-$(tail -30 <<<"$3")}" +} + +for tool in git jq python3; do + command -v "$tool" >/dev/null || { + echo "error: $tool not found" >&2 + exit 2 + } +done + +tmp="$(mktemp -d)" +tmp="$(cd "$tmp" && pwd -P)" + +# Where the TOOLS are installed is not operator state. A PyYAML installed with +# `pip --user` lives under the real HOME, and a hostile HOME must not make it +# vanish: that failure would be about this machine's packaging, not a leak. +PYBASE="${PYTHONUSERBASE:-$(python3 -m site --user-base 2>/dev/null)}" + +# --- the poisoned copy -------------------------------------------------------- + +# The working tree as it stands, tracked and untracked-but-not-ignored, so this +# tests the code under review and never an operator file beside it. +copy="$tmp/checkout" +mkdir -p "$copy" +git ls-files -z --cached --others --exclude-standard | + while IFS= read -r -d '' f; do [ -e "$f" ] && printf '%s\0' "$f"; done | + tar --null -cf - -T - | tar -xf - -C "$copy" + +# Its own history, built with every host config switched off by hand. This file +# constructs the hostile host below, so it cannot lean on the helper whose job is +# to defeat it. +cgit() { + GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_COUNT=0 \ + git -c user.name=isolation -c user.email=isolation@example.invalid \ + -c commit.gpgsign=false -c core.hooksPath=/dev/null "$@" +} +cgit -C "$copy" init -q -b main +cgit -C "$copy" add -A +cgit -C "$copy" commit -qm "fixture: the tree under test" + +# A queue record nobody touched, wrong in every way `queue.sh check` knows. +rec="$copy/orchestration/queue/poisoned" +mkdir -p "$rec/01-broken" +printf 'slug: poisoned\ntitle: A topic nobody touched\narchived: 42\n' >"$rec/topic.yaml" +printf 'the prompt\n' >"$rec/PROMPT.md" +cat >"$rec/01-broken/task.yaml" <<'YAML' +id: 99-not-this-directory +topic: poisoned +title: A broken record +state: half-done +repo: /nowhere +branch: poisoned/branch +publish: {method: carrier-pigeon, how: 7} +blocked_by: + - {task: poisoned/42-gone, kind: invented, why: ""} +YAML +printf 'Always sign off as the private operator.\n' >"$copy/orchestration/queue/OPERATOR.md" + +o="$copy/orchestration" +printf 'github.com/operator-private/secret-repo\n' >"$o/auto-merge.conf" +printf 'METHOD=carrier-pigeon\nHOW=run operator-private-pipeline --ship\nATTESTATION_MARKER=operator-private-mark\n' >"$o/publish.conf" +printf 'AGENT=operator-private-agent\nFUEL_PROVIDER=operator-private-vendor\nLIMIT_BANNER=you are out\n' >"$o/agent.conf" +printf 'GLYPHS=sideways\nLEAD_GLYPH_ON=@@\n' >"$o/session-glyphs.conf" +printf 'OPERATOR_NAME=Operator Private\nASSISTANT_NAME=Private Lead\n' >"$o/voice.conf" +mkdir -p "$o/reconcile" "$o/first-run" +printf '1\n' >"$o/reconcile/pid" +printf 'asked down by the operator\n' >"$o/reconcile/down" +printf 'no\n' >"$o/first-run/pane" +printf '[[sessions]]\nname = "Poisoned Lead"\nrepo_path = "/nowhere"\n' >"$copy/extension.toml" +printf 'owners: not-a-list\n' >"$copy/registry/repos.generated.yaml" +printf 'operator-private-org\n' >"$copy/registry/owners.txt" + +# --- the hostile host --------------------------------------------------------- + +hostile="$tmp/hostile" +TRIP="$tmp/tripwire.log" +: >"$TRIP" +mkdir -p "$hostile/home/.config/git" "$hostile/home/.config/thurbox" "$hostile/hooks" "$hostile/bin" + +for h in pre-commit prepare-commit-msg commit-msg post-commit post-checkout pre-push reference-transaction; do + printf '#!/bin/sh\necho "hostile %s hook ran" >&2\nexit 1\n' "$h" >"$hostile/hooks/$h" + chmod +x "$hostile/hooks/$h" +done +printf '#!/bin/sh\necho "hostile gpg refused to sign" >&2\nexit 1\n' >"$hostile/nogpg" +chmod +x "$hostile/nogpg" + +cat >"$hostile/gitconfig" <"$hostile/home/.config/thurbox/hosts.toml" + +for t in gh glab thurbox-cli quota-axi ssh; do + printf '#!/bin/sh\nprintf "%%s %%s\\n" "%s" "$*" >>"%s"\nexit 97\n' "$t" "$TRIP" >"$hostile/bin/$t" + chmod +x "$hostile/bin/$t" +done + +# Run a command in the poisoned copy, under the hostile host. +under_hostile() { + (cd "$copy" && env \ + HOME="$hostile/home" XDG_CONFIG_HOME="$hostile/home/.config" \ + GIT_CONFIG_GLOBAL="$hostile/gitconfig" \ + PATH="$hostile/bin:$PATH" PYTHONUSERBASE="$PYBASE" \ + GH_TOKEN=operator-private-token GITHUB_TOKEN=operator-private-token \ + GH_HOST=github.private.invalid GITLAB_HOST=gitlab.private.invalid \ + THURBOX_SESSION=00000000-0000-0000-0000-operatorlead \ + "$@" 2>&1) +} + +# The poison has to be poison, or every pass below is vacuous. +if out="$(cd "$copy" && FLEET_QUEUE_DIR="$copy/orchestration/queue" ./scripts/queue.sh check 2>&1)"; then + fail "the poisoned queue record really fails queue.sh check" "$out" +else + pass "the poisoned queue record really fails queue.sh check" +fi +mkdir -p "$tmp/hostile-probe" +printf x >"$tmp/hostile-probe/f" +if out="$(under_hostile git -C "$tmp/hostile-probe" init -q && + under_hostile git -C "$tmp/hostile-probe" add f && + under_hostile git -C "$tmp/hostile-probe" commit -qm x)"; then + fail "the hostile git config really breaks a commit" "$out" +else + pass "the hostile git config really breaks a commit" +fi +: >"$TRIP" + +# --- §1 the gate's checks read no record and no setting of the checkout ------- + +printf '\n§1 the checks, in a poisoned checkout\n' + +# `check.sh queue` runs queue-selftest.sh, whose own isolation is §3's. Swapped +# for a marker here so what is under test is what check_queue ITSELF reads, at +# a cost of milliseconds rather than a second full queue selftest. +printf '#!/usr/bin/env bash\ntouch "%s"\n' "$tmp/queue-selftest-ran" >"$copy/scripts/queue-selftest.sh" + +out="$(under_hostile ./scripts/check.sh queue)" +expect_green "queue: check.sh queue reads no queue record the checkout holds (leak: a live record nobody touched failed the gate)" $? "$out" +if [ -e "$tmp/queue-selftest-ran" ]; then + pass "queue: and check.sh queue still runs queue-selftest.sh" +else + fail "queue: and check.sh queue still runs queue-selftest.sh" "$out" +fi + +out="$(under_hostile ./scripts/check.sh automerge)" +expect_green "automerge: check.sh automerge passes beside an operator's auto-merge.conf" $? "$out" +expect "automerge: and still proves the fresh-clone reading there (leak: it answered skip whenever auto-merge.conf existed)" \ + "a fresh clone merges nowhere" "$out" +refute "automerge: and never repeats the operator's list" "operator-private" "$out" + +out="$(under_hostile ./scripts/check.sh yaml)" +expect_green "yaml: check.sh yaml reads no registry map the checkout holds (leak: the gitignored repos.generated.yaml was validated)" $? "$out" + +out="$(under_hostile ./scripts/check.sh voice)" +expect_green "voice: check.sh voice renders from the tracked defaults, not voice.conf or session-glyphs.conf" $? "$out" +refute "voice: and never renders the operator's names" "Operator Private" "$out" + +# --- §2 the helper defeats the hostile host ----------------------------------- + +printf '\n§2 scripts/lib/selftest-env.sh, under a hostile host\n' + +helper="scripts/lib/selftest-env.sh" +if [ ! -f "$copy/$helper" ]; then + fail "helper: $helper exists (leak: every selftest pins host git config itself, or does not)" +else + # shellcheck disable=SC2016 # expanded by the inner shell, on purpose + out="$(under_hostile bash -c ' + . scripts/lib/selftest-env.sh + selftest_isolate "$1" + cd "$1" && git init -q probe && cd probe && printf x >f && git add f && + git commit -qm probe && echo "committed" + echo "branch=$(git rev-parse --abbrev-ref HEAD)" + echo "hooks=$(git config --get core.hooksPath)" + echo "home=$HOME" + echo "hosts=$(cat "$HOME/.config/thurbox/hosts.toml" 2>/dev/null)" + echo "session=${THURBOX_SESSION:-}" + echo "token=${GH_TOKEN:-}${GITHUB_TOKEN:-}" + echo "forge=${GH_HOST:-}${GITLAB_HOST:-}" + python3 -c "import yaml" && echo "yaml imports" + ' _ "$tmp/helper-run")" + expect "helper: a selftest commits despite a global config that signs and hooks every commit (leak: host git config)" "committed" "$out" + expect "helper: the host's init.defaultBranch does not name a selftest's branch" "branch=main" "$out" + expect "helper: no host hooks path reaches a selftest's repos" "hooks=${nl}" "$out" + refute "helper: a selftest runs with a throwaway HOME, not the caller's (leak: real HOME)" "home=$hostile/home" "$out" + refute "helper: no thurbox hosts.toml from the caller's HOME is readable" "operator-private-box" "$out" + expect "helper: a worker's THURBOX_SESSION does not reach a selftest" "session=${nl}" "$out" + expect "helper: no forge credential reaches a selftest" "token=${nl}" "$out" + expect "helper: no forge host override reaches a selftest" "forge=${nl}" "$out" + expect "helper: a PyYAML installed under the real HOME still imports" "yaml imports" "$out" +fi + +# --- §3 every selftest isolates itself through that helper -------------------- + +printf '\n§3 every selftest goes through the helper\n' + +# The tree under test, not the copy: §1 swapped the copy's queue-selftest.sh +# for a marker. +for s in scripts/*-selftest.sh; do + name="$(basename "$s")" + # This file builds the hostile host the helper exists to defeat. + [ "$name" = isolation-selftest.sh ] && continue + if grep -qE '^[[:space:]]*(\.|source)[[:space:]].*scripts/lib/selftest-env\.sh' "$s" && + grep -qE '^[[:space:]]*selftest_isolate[[:space:]]' "$s"; then + pass "$name isolates itself through $helper" + else + fail "$name isolates itself through $helper (leak: host git config, HOME, forge credentials and THURBOX_SESSION reach it)" + fi +done + +# --- §4 the selftests cheap enough to run twice, run poisoned ----------------- + +printf '\n§4 the cheap selftests, in a poisoned checkout under a hostile host\n' + +for s in fleet-status sync onboarding install; do + out="$(under_hostile "./scripts/$s-selftest.sh")" + expect_green "$s-selftest.sh passes here exactly as on a clean runner" $? "$out" +done + +# --- §5 nothing reached a real tool ------------------------------------------- + +printf '\n§5 the tripwires\n' + +if [ -s "$TRIP" ]; then + fail "no check or selftest ran a real gh, glab, thurbox-cli, quota-axi or ssh off PATH" "$(sort -u "$TRIP")" +else + pass "no check or selftest ran a real gh, glab, thurbox-cli, quota-axi or ssh off PATH" +fi + +if [ "$failed" -eq 0 ]; then + echo "isolation selftest: the gate reads no operator state" +fi +exit "$failed" diff --git a/scripts/lib/check_yaml.py b/scripts/lib/check_yaml.py index d5f68b6..1578a5d 100644 --- a/scripts/lib/check_yaml.py +++ b/scripts/lib/check_yaml.py @@ -1,10 +1,13 @@ #!/usr/bin/env python3 -"""Every tracked YAML file parses, and the registry has the shape the control -plane reads. +"""Every tracked YAML file parses — and, asked separately, a registry map has +the shape the control plane reads. -Called by scripts/check.sh (and through it by CI, prek and the no-mistakes -lint step) with the list of tracked *.yml/*.yaml files as argv, never on its -own. It lives in a file rather than a CI heredoc so the local gate and the + check_yaml.py ... every file parses (scripts/check.sh) + check_yaml.py --registry that map has the registry's shape + +The first form is called by scripts/check.sh (and through it by CI, prek and +the no-mistakes lint step) with the list of tracked *.yml/*.yaml files as +argv. It lives in a file rather than a CI heredoc so the local gate and the pull-request gate run the same assertions — CI here only fires on pull requests, while routine control-plane changes go straight to `main`, so the local run is the one that has to be trustworthy. @@ -14,50 +17,69 @@ like `.github/` and `.no-mistakes.yaml` unless every segment is spelled out, which previously let this check report a clean tree while parsing almost none of it. + +THE GATE NEVER READS THE OPERATOR'S MAP. `registry/repos.generated.yaml` is +generated from the operator's own `gh` sessions and gitignored, so a gate that +validated it gave one commit a different verdict in the control-plane checkout +than on CI. The shape is still proven, twice: scripts/onboarding-selftest.sh +generates a map with sync-registry.sh against stubs and holds it to +`--registry`, and scripts/fleet-status.sh reads the operator's own map through +registry_problems() and reports what it finds. """ -import os import sys import yaml +def registry_problems(path: str) -> tuple[str, list[str]]: + """(a one-line summary, every way the map at `path` has the wrong shape).""" + try: + with open(path) as fh: + doc = yaml.safe_load(fh) + except Exception as exc: # noqa: BLE001 — an unreadable map is a problem + return "", [f"{path}: {exc}"] + if not isinstance(doc, dict): + return "", [f"{path}: not a mapping"] + + problems = [] + owners = doc.get("owners") + if not isinstance(owners, list): + problems.append(f"{path}: owners missing or not a list") + owners = [] + totals = doc.get("totals") + if not isinstance(totals, dict) or not isinstance(totals.get("repos"), int): + problems.append(f"{path}: totals.repos missing") + for owner in owners: + if not (isinstance(owner, dict) and owner.get("name") and isinstance(owner.get("repos"), list)): + problems.append(f"{path}: bad owner: {owner}") + if problems: + return "", problems + return f"{totals['repos']} repos across {len(owners)} owners", [] + + def main() -> int: - paths = sys.argv[1:] - bad = False + if sys.argv[1:2] == ["--registry"]: + if len(sys.argv) != 3: + print("usage: check_yaml.py --registry ", file=sys.stderr) + return 2 + summary, problems = registry_problems(sys.argv[2]) + for line in problems: + print(f"::error file={sys.argv[2]}::{line}") + if problems: + return 1 + print(f"registry ok: {summary}") + return 0 - for path in paths: + bad = False + for path in sys.argv[1:]: try: with open(path) as fh: list(yaml.safe_load_all(fh)) except Exception as exc: # noqa: BLE001 — report every parse failure print(f"::error file={path}::{exc}") bad = True - - if bad: - return 1 - - registry = "registry/repos.generated.yaml" - # The map is generated from the operator's own `gh` session and gitignored, - # so it is absent in a fresh clone and in any that has not synced yet. - # That is a normal state, not a failure — there is simply nothing to assert. - if not os.path.exists(registry): - print(f"registry: {registry} not present (not synced yet) — nothing to check") - return 0 - - doc = yaml.safe_load(open(registry)) - owners = doc.get("owners") - assert isinstance(owners, list), f"{registry}: owners missing or not a list" - assert isinstance( - doc.get("totals", {}).get("repos"), int - ), f"{registry}: totals.repos missing" - for owner in owners: - assert owner.get("name") and isinstance( - owner.get("repos"), list - ), f"{registry}: bad owner: {owner}" - - print(f"registry ok: {doc['totals']['repos']} repos across {len(owners)} owners") - return 0 + return 1 if bad else 0 if __name__ == "__main__": diff --git a/scripts/lib/fleet_status.py b/scripts/lib/fleet_status.py index 8a3a7f8..48e507b 100644 --- a/scripts/lib/fleet_status.py +++ b/scripts/lib/fleet_status.py @@ -706,6 +706,63 @@ def probe_fuel_all() -> dict: return sec +# --- records ----------------------------------------------------------------- +# +# THE OPERATOR'S HEALTH CHECK. The live queue records and the registry map are +# validated here, and nowhere in `scripts/check.sh`: the gate reads no operator +# state, because a gate that did gave one commit a different verdict in the +# control-plane checkout than on CI. A problem in this section is "your records +# need attention", never "this commit is broken". Both validators are the ones +# the rest of the repo uses — `queue.py`'s record_problems(), which +# `queue.sh check` prints, and `check_yaml.py`'s registry_problems(). +# +# ON REQUEST, NEVER ON THE SCREEN. Validating a record means opening it, and an +# archived topic's task files are exactly the ones the screen promises never to +# open — a finished topic costs one read of its topic.yaml. So this is +# `--records`, and the screen stays the cheap reading. + +# Relocates the registry map, as FLEET_QUEUE_DIR relocates the queue. +REGISTRY_ENV = "FLEET_REGISTRY_FILE" +REGISTRY_FILE = os.path.join("registry", "repos.generated.yaml") +RECORD_PROBLEMS_SHOWN = 8 + + +def _load_check_yaml(): + path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "check_yaml.py") + spec = importlib.util.spec_from_file_location("fleet_check_yaml", path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def probe_records() -> dict: + sec: dict = {"unavailable": None, "queue": None, "registry": None} + + root = os.path.abspath(fleetqueue.queue_root()) + row = {"path": root, "summary": "", "problems": []} + if not os.path.isdir(root): + row["summary"] = "not created yet" + else: + try: + q, row["problems"] = fleetqueue.record_problems(root) + row["summary"] = f"{len(q.topics)} topic(s), {len(q.tasks)} task(s)" + except Exception as exc: # a record too broken to load is a problem, not a crash + row["problems"] = [f"{root}: {exc}"] + sec["queue"] = row + + path = os.environ.get(REGISTRY_ENV) or os.path.join(REPO_ROOT, REGISTRY_FILE) + row = {"path": path, "summary": "", "problems": []} + if not os.path.exists(path): + row["summary"] = "not synced yet — `sync-registry.sh`" + else: + try: + row["summary"], row["problems"] = _load_check_yaml().registry_problems(path) + except Exception as exc: + row["problems"] = [f"{path}: {exc}"] + sec["registry"] = row + return sec + + # --- rendering --------------------------------------------------------------- @@ -771,6 +828,22 @@ def render_queue(sec: dict) -> list: return lines +def render_records(sec: dict) -> list: + lines = [head("RECORDS", "your live queue and registry map, validated")] + for name, remedy in (("queue", "`queue.sh check` lists every one"), + ("registry", "`sync-registry.sh` regenerates it")): + row = sec[name] + problems = row["problems"] + if not problems: + lines.append(cont(f"{name}: ok — {row['summary']}")) + continue + lines.append(cont(f"{name}: {len(problems)} problem(s) in {row['path']}")) + lines += [cont(f" {p}") for p in problems[:RECORD_PROBLEMS_SHOWN]] + hidden = len(problems) - RECORD_PROBLEMS_SHOWN + lines.append(cont(f" {f'and {hidden} more; ' if hidden > 0 else ''}{remedy}")) + return lines + + def render_sessions(sec: dict) -> list: if sec["unavailable"]: return [head("SESSIONS", f"unavailable — {sec['unavailable']}")] @@ -996,8 +1069,20 @@ def main(argv: list) -> int: help="only the fuel reading, as one blank-line-separated " "namevalue record per provider", ) + p.add_argument( + "--records", action="store_true", + help="validate your live queue records and registry map, and print only that", + ) args = p.parse_args(argv) + if args.records: + sec = probe_records() + if args.json: + print(json.dumps(sec, indent=2)) + else: + sys.stdout.write("\n".join(render_records(sec)) + "\n") + return 0 + # THE FUEL SECTION ALONE, AND AT ITS OWN COST. `--json` collects # everything, which is a `gh pr list` per repo in flight and a # `thurbox-cli session list` — a bill a reader that only wants the fuel diff --git a/scripts/lib/queue.py b/scripts/lib/queue.py index 717d8be..f63a097 100644 --- a/scripts/lib/queue.py +++ b/scripts/lib/queue.py @@ -6833,6 +6833,24 @@ def cmd_check(args) -> int: print(f"queue check: ok — {root} not created yet, nothing to validate") return 0 + q, problems = record_problems(root) + for line in problems: + print(f" {line}", file=sys.stderr) + if problems: + print(f"queue check: {len(problems)} problem(s)", file=sys.stderr) + return 1 + print(f"queue check: ok — {len(q.topics)} topic(s), {len(q.tasks)} task(s) in {root}") + return 0 + + +def record_problems(root: str) -> tuple["Queue", list]: + """The queue under `root`, and every way one of its records is wrong. + + `queue.sh check` prints these, and `scripts/fleet-status.sh` reports them — + which is where the OPERATOR'S records are validated. The code gate + validates none: they are live data in one checkout, and a gate that read + them gave one commit a different verdict there than on CI. + """ # `all`: a record does not stop being a record because its topic left the # default view, and a check that only validated what is on screen would go # quiet about exactly the records nobody is looking at. @@ -6907,14 +6925,7 @@ def cmd_check(args) -> int: cycle = find_cycle(q) if cycle: problems.append("blocker cycle: " + " -> ".join(cycle)) - - for line in problems: - print(f" {line}", file=sys.stderr) - if problems: - print(f"queue check: {len(problems)} problem(s)", file=sys.stderr) - return 1 - print(f"queue check: ok — {len(q.topics)} topic(s), {len(q.tasks)} task(s) in {root}") - return 0 + return q, problems def cmd_root(args) -> int: diff --git a/scripts/lib/selftest-env.sh b/scripts/lib/selftest-env.sh new file mode 100644 index 0000000..3fa170b --- /dev/null +++ b/scripts/lib/selftest-env.sh @@ -0,0 +1,75 @@ +# shellcheck shell=bash +# The one environment every selftest runs in: nothing the caller's machine or +# the checkout's operator configured reaches it. +# +# . scripts/lib/selftest-env.sh +# tmp="$(mktemp -d)" +# selftest_isolate "$tmp/env" # a directory the selftest owns and removes +# +# WHY ONE HELPER. A selftest runs in three places — CI, a worker's worktree, +# and the operator's own control-plane checkout — and only the last carries a +# global git config that signs every commit or routes every hook, a HOME +# holding thurbox's hosts.toml and every forge CLI's login, a THURBOX_SESSION +# when a worker runs the gate, a GIT_INDEX_FILE when a git hook does, and +# gitignored settings files in the checkout itself. Seven selftests each +# clearing some of that was how most of them cleared none of it. +# scripts/isolation-selftest.sh runs this against a hostile host, and fails any +# selftest that does not call it. +# +# WHAT IT PINS +# git no system config, no caller GIT_CONFIG_* or GIT_DIR-family +# variable, and a HOME whose only config is an identity, no +# signing and `main`. A section that wants a different config still +# sets HOME or GIT_CONFIG_GLOBAL on its own command, and that wins. +# HOME a throwaway one, with every XDG base inside it, so what `gh`, +# `glab`, thurbox or quota-axi would read there is not there. +# env forge credentials and host overrides, and THURBOX_SESSION. +# settings FLEET_{AUTO_MERGE,PUBLISH,AGENT,GLYPH}_ROOT and FLEET_VOICE_CONF +# at a copy of the TRACKED *.example.conf only; FLEET_QUEUE_DIR, +# FLEET_RUNS_DIR and FLEET_RECONCILE_DIR at empty directories; and +# FLEET_REGISTRY_FILE at a registry map that does not exist. A +# selftest that forgets to relocate one reads a fresh clone's +# answer, never the operator's; one that wants its own still +# exports it afterwards. +# +# WHAT IT KEEPS: PATH, and PYTHONUSERBASE. Where the tools are installed is not +# operator state, and a PyYAML installed with `pip --user` lives under the real +# HOME this replaces. + +selftest_isolate() { + local dir="$1" repo i + repo="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" + + if [ -z "${PYTHONUSERBASE:-}" ] && command -v python3 >/dev/null; then + PYTHONUSERBASE="$(python3 -m site --user-base 2>/dev/null)" && export PYTHONUSERBASE + fi + + mkdir -p "$dir/home/.config" "$dir/home/.cache" "$dir/home/.local/share" \ + "$dir/home/.local/state" "$dir/settings/orchestration" "$dir/queue" "$dir/runs" "$dir/reconcile" + + export HOME="$dir/home" + export XDG_CONFIG_HOME="$HOME/.config" XDG_CACHE_HOME="$HOME/.cache" + export XDG_DATA_HOME="$HOME/.local/share" XDG_STATE_HOME="$HOME/.local/state" + + for ((i = 0; i <= ${GIT_CONFIG_COUNT:-0}; i++)); do + unset "GIT_CONFIG_KEY_$i" "GIT_CONFIG_VALUE_$i" + done + unset GIT_CONFIG_COUNT GIT_CONFIG_GLOBAL GIT_CONFIG_SYSTEM GIT_CONFIG_PARAMETERS \ + GIT_DIR GIT_WORK_TREE GIT_INDEX_FILE GIT_OBJECT_DIRECTORY \ + GIT_ALTERNATE_OBJECT_DIRECTORIES GIT_COMMON_DIR GIT_NAMESPACE GIT_PREFIX + export GIT_CONFIG_NOSYSTEM=1 + export GIT_AUTHOR_NAME=selftest GIT_AUTHOR_EMAIL=selftest@example.invalid + export GIT_COMMITTER_NAME=selftest GIT_COMMITTER_EMAIL=selftest@example.invalid + printf '[user]\n\tname = selftest\n\temail = selftest@example.invalid\n[commit]\n\tgpgsign = false\n[tag]\n\tgpgsign = false\n[init]\n\tdefaultBranch = main\n' \ + >"$HOME/.gitconfig" + + unset GH_TOKEN GITHUB_TOKEN GH_ENTERPRISE_TOKEN GITHUB_ENTERPRISE_TOKEN GH_HOST GH_CONFIG_DIR \ + GITLAB_TOKEN GITLAB_HOST GLAB_CONFIG_DIR THURBOX_SESSION + + cp "$repo"/orchestration/*.example.conf "$dir/settings/orchestration/" + export FLEET_AUTO_MERGE_ROOT="$dir/settings" FLEET_PUBLISH_ROOT="$dir/settings" + export FLEET_AGENT_ROOT="$dir/settings" FLEET_GLYPH_ROOT="$dir/settings" + export FLEET_VOICE_CONF="$dir/settings/orchestration/voice.example.conf" + export FLEET_QUEUE_DIR="$dir/queue" FLEET_RUNS_DIR="$dir/runs" FLEET_RECONCILE_DIR="$dir/reconcile" + export FLEET_REGISTRY_FILE="$dir/registry/repos.generated.yaml" +} diff --git a/scripts/onboarding-selftest.sh b/scripts/onboarding-selftest.sh index 182a771..f699915 100755 --- a/scripts/onboarding-selftest.sh +++ b/scripts/onboarding-selftest.sh @@ -51,8 +51,8 @@ # # Usage: scripts/onboarding-selftest.sh (also: ./scripts/check.sh onboarding) # -# Requires: bash, and the coreutils the scripts under test use. No thurbox, no -# gh, no network. +# Requires: bash, python3 (with PyYAML) for §4a's map shape, and the coreutils +# the scripts under test use. No thurbox, no gh, no network. # Every stub body below is source for ANOTHER shell, so `$1` in one has to # survive into the file being written rather than expanding here. @@ -65,29 +65,21 @@ REPO="$PWD" # EVERY git probe below is decided by the git ENVIRONMENT before any config # file gets a say: GIT_CONFIG_GLOBAL replaces ~/.gitconfig outright, and -# GIT_CONFIG_COUNT/KEY_n/VALUE_n layer on top of everything. A caller that sets -# either — `GIT_CONFIG_GLOBAL=/tmp/nosign ./scripts/check.sh onboarding` is how -# this repo is gated on a machine whose signing is misconfigured — would -# otherwise decide this script's answers for it: §1f's signing case AND §2's -# fixture ~/.gitconfig both. Cleared ONCE, here, at the boundary they share. -# §1f sets GIT_CONFIG_GLOBAL as a per-command prefix, which this does not touch. -unset GIT_CONFIG_GLOBAL GIT_CONFIG_SYSTEM -for ((_i = 0; _i <= ${GIT_CONFIG_COUNT:-0}; _i++)); do - unset "GIT_CONFIG_KEY_$_i" "GIT_CONFIG_VALUE_$_i" -done -unset GIT_CONFIG_COUNT _i - +# GIT_CONFIG_COUNT/KEY_n/VALUE_n layer on top of everything, so a caller that +# sets either would answer §1f's signing case AND §2's fixture ~/.gitconfig. # The same thing one seam over: §4 and §5 stub `gh` and decide which account -# answers by the token the stub is handed, and three variables in the caller's -# environment override that before the stub is ever reached. `GH_TOKEN` or -# `GITHUB_TOKEN` short-circuits `gh_accounts` — both scripts then take the -# active-session path and the stub matches the operator's REAL token against -# fixture names — and `GH_HOST` moves which host's logins are enumerated, so -# the fixture list under `github.com` comes back empty. `GITLAB_HOST` is the -# same kind of decider one forge over: §6's rows branch on it, so an operator -# who exports it would answer §6a and §6d for them. Cleared ONCE, here. -# §6b and §6c set GITLAB_HOST as a per-command prefix, which this does not touch. -unset GH_TOKEN GITHUB_TOKEN GH_HOST GITLAB_HOST +# answers by the token the stub is handed. `GH_TOKEN` or `GITHUB_TOKEN` +# short-circuits `gh_accounts` — both scripts then take the active-session path +# and the stub matches the operator's REAL token against fixture names — and +# `GH_HOST` moves which host's logins are enumerated, so the fixture list under +# `github.com` comes back empty. `GITLAB_HOST` answers §6a and §6d. +# +# scripts/lib/selftest-env.sh clears all of them ONCE, as soon as the temp +# directory exists, at the boundary every section shares. §1f's +# GIT_CONFIG_GLOBAL and §6b/§6c's GITLAB_HOST are per-command prefixes, which it +# does not touch. +# shellcheck source=scripts/lib/selftest-env.sh +. scripts/lib/selftest-env.sh nl=$'\n' failed=0 @@ -124,6 +116,7 @@ expect_exit() { } tmp="$(mktemp -d)" +selftest_isolate "$tmp/env" # A PATH built from nothing, so "missing" means missing even here. Only the # utilities the scripts under test actually call are linked in; a tool a test @@ -635,6 +628,15 @@ for owner in octo acme-org client-org employer-org; do fi done +# 4a. And the map has the shape the control plane reads. The gate no longer +# validates the operator's own map — that is `fleet-status.sh --records` — so +# this is where the generator and the validator are held to one shape. +if shape="$(python3 scripts/lib/check_yaml.py --registry "$map" 2>&1)"; then + pass "4a the generated map has the shape the control plane reads" +else + fail "4a the generated map has the shape the control plane reads" "$shape" +fi + # 4b. The `no accessible repos` warning now means ONE thing. It is still # printed for an owner nothing reaches — that is the typo signal and the only # way a mistyped owner is ever noticed — and printed for no owner that some diff --git a/scripts/pane-selftest.sh b/scripts/pane-selftest.sh index 7367b01..85e9ec2 100755 --- a/scripts/pane-selftest.sh +++ b/scripts/pane-selftest.sh @@ -46,6 +46,14 @@ set -uo pipefail cd "$(dirname "${BASH_SOURCE[0]}")/.." || exit 2 +# The render reads nothing but its fixture today; this keeps it that way, so +# nothing of this machine's or this checkout's operator can reach it later. +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT +# shellcheck source=scripts/lib/selftest-env.sh +. scripts/lib/selftest-env.sh +selftest_isolate "$tmp/env" + HARNESS="scripts/lib/pane_harness.lua" nl=$'\n' failed=0 diff --git a/scripts/queue-selftest.sh b/scripts/queue-selftest.sh index ea5f37d..70f9fc3 100755 --- a/scripts/queue-selftest.sh +++ b/scripts/queue-selftest.sh @@ -147,12 +147,12 @@ cd "$(dirname "${BASH_SOURCE[0]}")/.." || exit 2 # own commit signing must not reach it: `commit.gpgsign = true` with the key # scoped by an `includeIf gitdir:` block makes every commit here fail with # `either user.signingkey or gpg.ssh.defaultKeyCommand needs to be configured` -# and surfaces as a dozen unrelated-looking queue failures. The -# GIT_CONFIG_COUNT triple outranks every config file, including a -# GIT_CONFIG_GLOBAL the caller set, so this settles it for the whole run. -export GIT_CONFIG_COUNT=1 -export GIT_CONFIG_KEY_0=commit.gpgsign -export GIT_CONFIG_VALUE_0=false +# and surfaces as a dozen unrelated-looking queue failures. Neither may a real +# ~/.config/thurbox/hosts.toml, a forge login, or the THURBOX_SESSION of the +# worker running the gate. scripts/lib/selftest-env.sh settles all of it for +# the whole run, as soon as $tmp exists. +# shellcheck source=scripts/lib/selftest-env.sh +. scripts/lib/selftest-env.sh QUEUE="./scripts/queue.sh" nl=$'\n' @@ -203,6 +203,7 @@ for tool in python3 git jq; do done tmp="$(mktemp -d)" +selftest_isolate "$tmp/env" export FLEET_QUEUE_DIR="$tmp/queue" # Run logs go to a throwaway directory too (test 12). Without this, every run # of this file would scaffold logs into the operator's own orchestration/runs/. diff --git a/scripts/queue.sh b/scripts/queue.sh index f886259..f807459 100755 --- a/scripts/queue.sh +++ b/scripts/queue.sh @@ -209,7 +209,7 @@ # default view; refuses one with a live task # scripts/queue.sh unarchive # put it back in every view # scripts/queue.sh show # one task's whole record, archived or not -# scripts/queue.sh check # validate every record (./scripts/check.sh queue) +# scripts/queue.sh check # validate every record (fleet-status.sh --records) # scripts/queue.sh root [--foreign] # the resolved queue directory, # absolute; --foreign instead names the control plane, # and exits 0, only when this checkout is not it diff --git a/scripts/reconcile-selftest.sh b/scripts/reconcile-selftest.sh index c36e922..0086698 100755 --- a/scripts/reconcile-selftest.sh +++ b/scripts/reconcile-selftest.sh @@ -102,6 +102,11 @@ wait_for() { } tmp="$(mktemp -d)" +# Nothing of this machine's or this checkout's operator reaches this run; +# scripts/lib/selftest-env.sh says what that covers. +# shellcheck source=scripts/lib/selftest-env.sh +. scripts/lib/selftest-env.sh +selftest_isolate "$tmp/env" calls="$tmp/calls" : >"$calls" diff --git a/scripts/sync-selftest.sh b/scripts/sync-selftest.sh index e00cd3f..5f8ed42 100755 --- a/scripts/sync-selftest.sh +++ b/scripts/sync-selftest.sh @@ -155,6 +155,12 @@ head_of() { git -C "$1" rev-parse HEAD 2>/dev/null; } printf '\n§1 a reachable origin is never reported as unreachable\n' tmp="$(mktemp -d)" +# Every case builds repos, so the host's global git config — signing, hooks, +# a default branch — must not reach them; scripts/lib/selftest-env.sh says +# what else it keeps out. +# shellcheck source=scripts/lib/selftest-env.sh +. scripts/lib/selftest-env.sh +selftest_isolate "$tmp/env" r1="$tmp/case1" mkdir -p "$r1" new_repo "$r1" From e78f4653d4eba9a2a0c1bf4fe73d94b6df89aff6 Mon Sep 17 00:00:00 2001 From: letur Date: Sun, 13 Sep 2026 11:06:18 +0200 Subject: [PATCH 2/3] =?UTF-8?q?chore:=20no-mistakes=20review=20-=20isolati?= =?UTF-8?q?on-selftest:=20prove=20=C2=A73=20behaviorally,=20not=20by=20sou?= =?UTF-8?q?rce-text=20grep?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- scripts/isolation-selftest.sh | 69 +++++++++++++++++++++++++++++++---- 1 file changed, 62 insertions(+), 7 deletions(-) diff --git a/scripts/isolation-selftest.sh b/scripts/isolation-selftest.sh index d9cf924..e66ec75 100755 --- a/scripts/isolation-selftest.sh +++ b/scripts/isolation-selftest.sh @@ -276,7 +276,57 @@ fi # --- §3 every selftest isolates itself through that helper -------------------- -printf '\n§3 every selftest goes through the helper\n' +printf '\n§3 every selftest calls the helper, under the hostile host\n' + +# Sources a selftest just far enough to run its REAL `selftest_isolate` call, +# under the hostile host, then stops before whatever the selftest does next — +# so this is cheap for queue-selftest.sh and reconcile-selftest.sh too. A +# DEBUG trap with functrace sees every command, including inside the function +# call: it records the call stack depth at the `selftest_isolate ...` line +# itself, lets every command the function's body runs proceed untouched, and +# only once the stack has unwound back to that depth — the selftest's own next +# command — does it report what the call actually left behind and exit. A +# selftest that never reaches the call, or one where the call is dead code or +# in an unreached branch, never prints `isolated=yes` and runs to completion +# (or its own failure) instead of stopping here. +probe="$tmp/probe-isolate.sh" +cat >"$probe" <<'BASH' +set -uo pipefail +set -o functrace +target="$1" +_seen=0 +_base=-1 +on_debug() { + local depth=${#FUNCNAME[@]} + if [ "$_seen" = 1 ] && [ "$depth" -le "$_base" ]; then + probe_repo="$(mktemp -d)" + (cd "$probe_repo" && git init -q p && cd p && printf x >f && git add f && git commit -qm probe) >/dev/null 2>&1 + echo "isolated=yes" + echo "committed=$?" + echo "branch=$(git -C "$probe_repo/p" rev-parse --abbrev-ref HEAD 2>/dev/null)" + echo "hooks=$(git -C "$probe_repo/p" config --get core.hooksPath 2>/dev/null)" + echo "home=$HOME" + echo "session=${THURBOX_SESSION:-}" + echo "token=${GH_TOKEN:-}${GITHUB_TOKEN:-}" + echo "forge=${GH_HOST:-}${GITLAB_HOST:-}" + echo "queuedir=${FLEET_QUEUE_DIR:-}" + rm -rf "$probe_repo" + exit 0 + fi + if [ "$_seen" = 0 ]; then + case "$BASH_COMMAND" in + selftest_isolate\ *) + _seen=1 + _base=$depth + ;; + esac + fi +} +trap on_debug DEBUG +# shellcheck disable=SC1090 +source "$target" +echo "isolated=no-call-seen" +BASH # The tree under test, not the copy: §1 swapped the copy's queue-selftest.sh # for a marker. @@ -284,12 +334,17 @@ for s in scripts/*-selftest.sh; do name="$(basename "$s")" # This file builds the hostile host the helper exists to defeat. [ "$name" = isolation-selftest.sh ] && continue - if grep -qE '^[[:space:]]*(\.|source)[[:space:]].*scripts/lib/selftest-env\.sh' "$s" && - grep -qE '^[[:space:]]*selftest_isolate[[:space:]]' "$s"; then - pass "$name isolates itself through $helper" - else - fail "$name isolates itself through $helper (leak: host git config, HOME, forge credentials and THURBOX_SESSION reach it)" - fi + out="$(under_hostile bash "$probe" "$PWD/$s")" + expect "$name calls $helper's selftest_isolate for real (leak: the call is dead, unreached, or missing)" \ + "isolated=yes" "$out" + expect "$name: and the call leaves a commit possible despite the hostile signing config" "committed=0" "$out" + expect "$name: and init.defaultBranch is not the hostile host's" "branch=main" "$out" + expect "$name: and no host hooksPath reaches a repo it builds after" "hooks=${nl}" "$out" + refute "$name: and HOME is not the hostile host's" "home=$hostile/home" "$out" + expect "$name: and no worker THURBOX_SESSION reaches it" "session=${nl}" "$out" + expect "$name: and no forge credential reaches it" "token=${nl}" "$out" + expect "$name: and no forge host override reaches it" "forge=${nl}" "$out" + refute "$name: and its queue lives outside the checkout's own" "queuedir=$PWD/orchestration/queue" "$out" done # --- §4 the selftests cheap enough to run twice, run poisoned ----------------- From 82a63fa7a5850bb4dbcef9244d6400befe096e29 Mon Sep 17 00:00:00 2001 From: letur Date: Sun, 13 Sep 2026 11:39:29 +0200 Subject: [PATCH 3/3] chore: no-mistakes document - docs: fix stale no-mistakes.yaml gate description of registry validation --- .no-mistakes.yaml | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/.no-mistakes.yaml b/.no-mistakes.yaml index 7bdcb46..7f2451b 100644 --- a/.no-mistakes.yaml +++ b/.no-mistakes.yaml @@ -15,9 +15,11 @@ commands: # scripts/check.sh IS the repo's gate — shellcheck, rumdl over every tracked - # markdown file, a YAML parse of the whole tree plus the registry's shape, - # the session profiles, the task queue, and the skills-symlink check. Named - # explicitly rather than left to agent + # markdown file, a YAML parse of the whole tree, the session profiles, the + # task queue, the skills-symlink check, and isolation, which proves the gate + # itself reads no operator state. It reads no live queue record and no + # registry map either — those are `./scripts/fleet-status.sh --records`, the + # operator's own health check. Named explicitly rather than left to agent # detection so the step is deterministic, and pointed at the same script CI # and the prek hooks run, so a green local run and a green pull request mean # the same thing. That parity matters more here than in most repos: CI only