Skip to content

fix(gate): make check.sh isolation-safe across CI, worktrees, and operator checkouts - #89

Merged
LeTuR merged 1 commit into
mainfrom
fix/test-isolation
Sep 13, 2026
Merged

fix(gate): make check.sh isolation-safe across CI, worktrees, and operator checkouts#89
LeTuR merged 1 commit into
mainfrom
fix/test-isolation

Conversation

@LeTuR

@LeTuR LeTuR commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Intent

Fleet task test-isolation/01-gate-reads-no-operator-state ("Fix isolation in fleet tests"; the Rust port is postponed). Goal: one commit gives the same ./scripts/check.sh result wherever it runs - CI, a worker worktree, or the operator's control-plane checkout with its live queue and gitignored configs. Leaks to confirm and close: (1) check_queue ran queue.sh check over the checkout's live orchestration/queue - moved out of the gate into an operator health check reachable from scripts/fleet-status.sh (implemented as fleet-status.sh --records, a flag rather than a screen section because validation opens archived topics' task files, which the default screen promises never to open; it validates both queue records via queue.py record_problems() and the registry map via check_yaml.py registry_problems(), and degrades/exits 0 like the rest of fleet-status). (2) check_automerge answered skip beside an operator auto-merge.conf - now judges a temp root holding only the tracked example. (3) selftests building git repos without pinning host git config - one shared helper scripts/lib/selftest-env.sh (selftest_isolate

) pins GIT_CONFIG_, GIT_DIR family, HOME/XDG, identity, forge credentials, THURBOX_SESSION and every FLEET_ settings root to tracked-example copies, keeping PATH and PYTHONUSERBASE; all seven selftests call it. (4) audit: check_yaml validated the gitignored registry map (removed from gate; onboarding-selftest now holds a generated map to the shape instead), check_voice rendered through the checkout's session-glyphs.conf/agent.conf (install-extension.sh now honours FLEET_GLYPH_ROOT), fleet-status-selftest's queue setup reached a real thurbox-cli (now runs under its bare PATH). Test first (operator rule): scripts/isolation-selftest.sh, wired as check.sh isolation and a CI job in All Checks, poisons a throwaway copy of the tree and a hostile host and re-runs only the checks that can leak plus the cheap selftests; §3 sources each selftest up to its real selftest_isolate call under the hostile host (so the two slow selftests are held to the helper without being re-run). It fails on main naming each leak and passes on the branch. Hard constraints: keep it simple, no test runner/sharding/new dependency; do not weaken what a check proves - move the live-data half, do not delete it; tracked files name no operator tool/vendor/agent; fixtures are made up; do not touch docs/design/testing.md (PR #85 owns it). Squash-merge only; the worker does not merge. This run is a follow-up on PR #88: the latest commit fixes §3's probe, which printed committed=$? after an echo and so always reported 0.

What Changed

  • Move live operator-state validation (queue records, registry map) out of check.sh and into a new fleet-status.sh --records health check backed by queue.py record_problems() and check_yaml.py registry_problems(); check_automerge now judges a temp root containing only the tracked auto-merge.conf example instead of the operator's live config.
  • Add scripts/lib/selftest-env.sh (selftest_isolate) to pin git config, GIT_DIR, HOME/XDG, identity, forge credentials, THURBOX_SESSION, and every FLEET_* root to tracked-example copies, and adopt it across all seven selftests (queue, install, fleet-status, onboarding, pane, sync, reconcile); install-extension.sh also gains FLEET_GLYPH_ROOT so glyph rendering can be isolated the same way.
  • Add scripts/isolation-selftest.sh, wired as check.sh isolation and a new CI job, which poisons a throwaway checkout and a hostile host and re-runs the leak-prone checks and selftests to prove the gate reads no operator state.

Risk Assessment

✅ Low: The target commit is a narrow, correct one-line fix (capturing the probe's exit status before it is clobbered by an intervening echo), and the surrounding branch content — already vetted in prior review rounds — consistently implements the stated goal (gate reads no operator state; live records moved to fleet-status.sh --records; automerge judged against a fresh-clone-only root; all seven selftests route through the shared selftest_isolate helper) with no source-content-only test assertions and no touch to docs/design/testing.md.

Testing

Ran ./scripts/isolation-selftest.sh directly (the smallest test that exercises this exact fix) and it passed cleanly end-to-end (all §1–§5 assertions green, exit 0). To prove the fix has teeth rather than just re-passing, I reproduced the pre-fix bug in isolation with two standalone scripts mirroring the buggy vs. fixed code under a hostile gpgsign-forcing git config: the buggy version always printed committed=0 regardless of the real commit outcome (echo resetting $?), while the fixed version correctly reported committed=128 when the commit genuinely failed. I then verified the fix in situ by temporarily reintroducing a real leak in scripts/lib/selftest-env.sh (letting GIT_CONFIG_GLOBAL survive selftest_isolate), re-ran isolation-selftest.sh, and confirmed the fixed §3 probe now correctly fails with committed=128 for every selftest instead of silently passing — then reverted the temporary change, confirmed git status is clean, and re-ran isolation-selftest.sh to confirm it is green again.

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • ./scripts/check.sh
  • ./scripts/isolation-selftest.sh (full run, post-fix, all green)
  • manual repro: buggy vs fixed committed=$? capture pattern under a gpgsign-forcing hostile git config
  • manual regression check: temporarily broke scripts/lib/selftest-env.sh's GIT_CONFIG_GLOBAL unset, re-ran ./scripts/isolation-selftest.sh, confirmed the fixed §3 probe now fails with committed=128 (detecting the reintroduced leak), then reverted and confirmed green + clean git status
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

§3's probe printed `committed=$?` after an `echo`, so it always read 0 and
the "a commit still works under hostile signing" assertion proved nothing.
Capture the status right after the commit; against a no-op selftest_isolate
the probe now reports committed=128.

Claude-Session: https://claude.ai/code/session_01TVf5c2rWYcoVREhFKgquNJ
@LeTuR
LeTuR force-pushed the fix/test-isolation branch from a656e44 to e91ed3d Compare September 13, 2026 10:10
@LeTuR
LeTuR merged commit e6ae710 into main Sep 13, 2026
12 checks passed
@LeTuR
LeTuR deleted the fix/test-isolation branch September 13, 2026 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant