Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,9 @@ jobs:
- name: Check postgres-contract still requires both job tokens
run: bash tests/release/postgres-contract-guard.test.sh

- name: Check local gate discovery and required-check warnings
run: bash tests/release/ci-local.test.sh

- name: Check the audit export states its confidentiality class
run: bash tests/release/audit-export-confidentiality.test.sh

Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ Releases before `0.2.5` predate the public launch; their notes live in the
- Align planner action descriptions with the Ubuntu execution fence, including
Ufw and the other tool families moved to shared or Debian-family support.
Guard both directions of the description/fence relationship (#416).
- Discover release/e2e shell guards in local CI instead of maintaining a stale
list. Report skipped required Postgres validation in the final summary and
prefer Podman when both container runtimes are available (#346).

## [0.15.0] — 2026-09-10

Expand Down
13 changes: 10 additions & 3 deletions docs/developer-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -387,12 +387,12 @@ cargo nextest run --workspace --locked

`scripts/ci-local.sh` mirrors the runnable jobs from
`.github/workflows/ci.yml` (rust, frontend, hygiene, security, and the
optional postgres-contract job) so you catch failures before pushing,
required postgres-contract job) so you catch failures before pushing,
without spending GitHub Actions minutes:

```sh
# Full run — everything CI runs, including the optional Postgres contract
# test if docker/podman is available (or SYSKNIFE_TEST_POSTGRES_URL is set)
# Full run — including the required Postgres contract when a URL or runtime
# is available (Podman is preferred over Docker)
scripts/ci-local.sh

# Fast subset — rust fmt/clippy/nextest + frontend tsc/vitest only
Expand All @@ -402,6 +402,13 @@ scripts/ci-local.sh --fast
scripts/ci-local.sh --no-postgres
```

The hygiene group discovers every `tests/release/*.test.sh` and
`tests/e2e/*.test.sh`; there are no deliberate exclusions. A missing Postgres
runtime/URL or `--no-postgres` produces a final **INCOMPLETE** warning naming
the required gate. Set `SYSKNIFE_TEST_POSTGRES_URL` or install Podman and rerun
without `--no-postgres` to satisfy it. Skips do not change the exit code; actual
check failures still exit nonzero.

It detects which tools are installed first: `cargo` and `node` are required
(missing either is a hard failure with an install link); an optional linter
that's missing (`cargo-nextest`, `cargo-audit`, `markdownlint-cli2`,
Expand Down
68 changes: 38 additions & 30 deletions scripts/ci-local.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,15 @@ are caught before pushing (and before spending GitHub Actions minutes).

--fast Rust fmt/clippy/nextest + frontend tsc/vitest only (the
same subset the pre-push hook runs)
--no-postgres Skip the optional postgres-contract job even when a
--no-postgres Skip the required postgres-contract job even when a
container runtime or SYSKNIFE_TEST_POSTGRES_URL is present
--install-hooks Set git core.hooksPath to .githooks (enables the pre-push
gate: scripts/ci-local.sh --fast) and exit -- does not
run any checks
--help Show this help and exit

Groups (default, full run): rust, frontend, hygiene, security,
postgres-contract (optional -- skipped if no runtime/URL is available).
postgres-contract (required in CI; a local skip leaves validation incomplete).

For a full Docker-based replay of the exact GitHub Actions workflow (all
jobs, exact runner image), see https://github.com/nektos/act instead.
Expand Down Expand Up @@ -76,6 +76,7 @@ cd "$repo_root"

RESULTS=()
hard_failures=0
required_skips=()

have() { command -v "$1" >/dev/null 2>&1; }

Expand Down Expand Up @@ -268,32 +269,27 @@ hygiene_shellcheck() (
| xargs -0 shellcheck --severity=warning
)

run_shell_tests() {
# No deliberate exclusions: every release/e2e *.test.sh is a local fixture
# or guard, not a live VM story. Keep any future exclusion here with its
# reason and an explicit SKIP outcome rather than maintaining a second list.
local test count=0
for test in "$repo_root"/tests/release/*.test.sh "$repo_root"/tests/e2e/*.test.sh; do
[[ -f "$test" ]] || continue
count=$((count + 1))
run_step "hygiene: ${test#"$repo_root/"}" bash "$test"
done
if ((count == 0)); then
record FAIL 'hygiene: no release/e2e shell tests discovered'
fi
}

run_hygiene_group() {
printf '\n### hygiene\n'
run_step 'hygiene: check_repo_completeness.sh' bash "$repo_root/scripts/check_repo_completeness.sh"
run_step 'hygiene: check_release_versions.sh' bash "$repo_root/scripts/check_release_versions.sh"
run_step 'hygiene: release-version-pins.test.sh' bash "$repo_root/tests/release/release-version-pins.test.sh"
run_step 'hygiene: public-claims.test.sh' bash "$repo_root/tests/release/public-claims.test.sh"
run_step 'hygiene: test-baseline-provenance.test.sh' bash "$repo_root/tests/release/test-baseline-provenance.test.sh"
run_step 'hygiene: npm test --prefix packages/setup' npm test --prefix "$repo_root/packages/setup"
run_step 'hygiene: registry-manifest.test.sh' bash "$repo_root/tests/release/registry-manifest.test.sh"
run_step 'hygiene: smithery-manifest.test.sh' bash "$repo_root/tests/release/smithery-manifest.test.sh"
run_step 'hygiene: release-rehearsal.test.sh' bash "$repo_root/tests/release/release-rehearsal.test.sh"
run_step 'hygiene: database-path-agreement.test.sh' bash "$repo_root/tests/release/database-path-agreement.test.sh"
run_step 'hygiene: node-eol.test.sh' bash "$repo_root/tests/release/node-eol.test.sh"
run_step 'hygiene: tracked-eol.test.sh' bash "$repo_root/tests/release/tracked-eol.test.sh"
run_step 'hygiene: systemd-directory-modes.test.sh' bash "$repo_root/tests/release/systemd-directory-modes.test.sh"
run_step 'hygiene: ubuntu-vm-bootstrap.test.sh' bash "$repo_root/tests/e2e/ubuntu-vm-bootstrap.test.sh"
run_step 'hygiene: provider-parity.test.sh' bash "$repo_root/tests/e2e/provider-parity.test.sh"
run_step 'hygiene: vm-env-secrets.test.sh' bash "$repo_root/tests/e2e/vm-env-secrets.test.sh"
run_step 'hygiene: story-metadata.test.sh' bash "$repo_root/tests/e2e/story-metadata.test.sh"
run_step 'hygiene: story-runner-verdicts.test.sh' bash "$repo_root/tests/e2e/story-runner-verdicts.test.sh"
run_step 'hygiene: vm-sync-parity.test.sh' bash "$repo_root/tests/e2e/vm-sync-parity.test.sh"
run_step 'hygiene: docs-share-cards.test.sh' bash "$repo_root/tests/release/docs-share-cards.test.sh"
run_step 'hygiene: install-paths.test.sh' bash "$repo_root/tests/release/install-paths.test.sh"
run_step 'hygiene: postgres-contract-guard.test.sh' bash "$repo_root/tests/release/postgres-contract-guard.test.sh"
run_step 'hygiene: audit-export-confidentiality.test.sh' bash "$repo_root/tests/release/audit-export-confidentiality.test.sh"
run_step 'hygiene: markdown-link-files.test.sh' bash "$repo_root/tests/release/markdown-link-files.test.sh"
run_shell_tests

if have markdownlint-cli2; then
run_step 'hygiene: markdownlint-cli2' hygiene_markdownlint
Expand Down Expand Up @@ -342,15 +338,16 @@ run_security_group() {
}

# ---------------------------------------------------------------------------
# postgres-contract (optional)
# postgres-contract (required in CI)
# ---------------------------------------------------------------------------

run_postgres_contract_group() {
printf '\n### postgres-contract (optional)\n'
printf '\n### postgres-contract (required in CI)\n'
local label="postgres-contract: live Postgres contract (store + CLI anchor exit code)"

if [[ "$run_postgres" != true ]]; then
record SKIP "${label} (--no-postgres)"
required_skips+=(postgres-contract)
return
fi

Expand All @@ -366,14 +363,15 @@ run_postgres_contract_group() {
fi

local runtime=""
if have docker; then
runtime="docker"
elif have podman; then
if have podman; then
runtime="podman"
elif have docker; then
runtime="docker"
fi

if [[ -z "$runtime" ]]; then
record SKIP "${label} (no SYSKNIFE_TEST_POSTGRES_URL and no docker/podman found)"
required_skips+=(postgres-contract)
return
fi

Expand Down Expand Up @@ -429,6 +427,7 @@ if [[ "$mode" == "full" ]]; then
run_postgres_contract_group
fi

print_summary() {
printf '\n=========================================\n'
printf ' ci-local summary (%s run)\n' "$mode"
printf '=========================================\n'
Expand All @@ -439,7 +438,16 @@ printf '=========================================\n'

if ((hard_failures > 0)); then
printf 'ci-local: FAIL (%d failing check(s))\n' "$hard_failures"
exit 1
fi

printf 'ci-local: PASS\n'
if ((${#required_skips[@]} > 0)); then
printf 'WARNING: REQUIRED CI check(s) did not run: %s\n' "${required_skips[*]}"
printf 'To run postgres-contract, set SYSKNIFE_TEST_POSTGRES_URL or install podman, then rerun without --no-postgres.\n'
printf 'ci-local: INCOMPLETE (required checks skipped)\n'
elif ((hard_failures == 0)); then
printf 'ci-local: PASS\n'
fi
}

print_summary
((hard_failures == 0))
116 changes: 116 additions & 0 deletions tests/release/ci-local.test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
#!/usr/bin/env bash
# Exercise the local gate orchestration without running privileged operations,
# installing dependencies, or recursively executing the discovered test scripts.
set -euo pipefail
cd "$(dirname "${BASH_SOURCE[0]}")/../.."
python3 - <<'PY'
import os
from pathlib import Path
import re
import subprocess
import tempfile
import unittest

ROOT = Path.cwd()
SOURCE = (ROOT / 'scripts/ci-local.sh').read_text(encoding='utf-8')


def functions(*names):
result = []
for name in names:
match = re.search(r'^' + name + r'\(\) \{\n.*?^\}', SOURCE, re.M | re.S)
if not match:
raise AssertionError('missing production function: ' + name)
result.append(match.group())
return '\n'.join(result)


def bash(code, root=ROOT):
# Pass a POSIX path through the invoking shell, including under Git Bash.
env = dict(os.environ, TEST_ROOT=root.as_posix())
return subprocess.run([os.environ.get('BASH_EXE', 'bash'), '-c', 'set -euo pipefail\nrepo_root="$TEST_ROOT"\n' + code],
env=env, text=True, capture_output=True)


class LocalGates(unittest.TestCase):
def test_hygiene_runs_every_discovered_and_ci_shell_test(self):
code = functions('run_hygiene_group')
# Include any extracted discovery function: the real hygiene entrypoint
# still decides whether to call it, so deleting that call fails this test.
if 'run_shell_tests() {' in SOURCE:
code += '\n' + functions('run_shell_tests')
code += '''
have() { return 1; }
record() { :; }
run_step() { shift; if [[ "${1:-}" == bash && "${2:-}" == *.test.sh ]]; then printf '%s\\n' "${2#"$repo_root/"}"; fi; }
run_hygiene_group
'''
result = bash(code)
self.assertEqual(result.returncode, 0, result.stderr)
actual = [line for line in result.stdout.splitlines() if line.startswith('tests/')]
disk = {str(p.relative_to(ROOT)).replace('\\', '/')
for d in ('release', 'e2e') for p in (ROOT / 'tests' / d).glob('*.test.sh')}
ci = set()
for workflow in (ROOT / '.github/workflows').glob('*.yml'):
ci.update(re.findall(r'tests/(?:release|e2e)/[\w-]+\.test\.sh', workflow.read_text(encoding='utf-8')))
self.assertTrue(disk, 'empty discovery must never pass')
self.assertTrue(ci, 'empty CI extraction must never pass')
self.assertEqual(set(actual), disk)
self.assertEqual(len(actual), len(set(actual)), 'a test must run exactly once')
self.assertFalse(ci - set(actual), 'CI tests missing locally: ' + str(ci - set(actual)))

def test_new_file_is_discovered_and_empty_directory_fails(self):
code = functions('record', 'run_shell_tests') + '''
RESULTS=(); hard_failures=0
run_step() { printf '%s\\n' "$*"; }
run_shell_tests
printf 'failures=%s\\n' "$hard_failures"
'''
with tempfile.TemporaryDirectory() as d:
root = Path(d)
for directory in ('release', 'e2e'):
(root / 'tests' / directory).mkdir(parents=True)
result = bash(code, root)
self.assertIn('failures=1', result.stdout)
(root / 'tests/release/future-guard.test.sh').touch()
result = bash(code, root)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn('future-guard.test.sh', result.stdout)
self.assertIn('failures=0', result.stdout)

def test_required_postgres_skip_is_in_final_summary(self):
code = functions('record', 'run_postgres_contract_group', 'print_summary')
code += '''
RESULTS=(); hard_failures=0; required_skips=(); mode=full
run_postgres=true
unset SYSKNIFE_TEST_POSTGRES_URL
have() { return 1; }
run_postgres_contract_group
print_summary
'''
result = bash(code)
self.assertEqual(result.returncode, 0, result.stderr)
summary = result.stdout.split('ci-local summary', 1)[1]
for text in ('REQUIRED', 'postgres-contract', 'SYSKNIFE_TEST_POSTGRES_URL', 'podman', 'INCOMPLETE'):
self.assertIn(text, summary)
self.assertNotIn('ci-local: PASS', summary)
explicit = bash(code.replace('run_postgres=true', 'run_postgres=false'))
self.assertIn('INCOMPLETE', explicit.stdout)

def test_podman_precedes_docker(self):
code = functions('record', 'run_postgres_contract_group') + '''
RESULTS=(); hard_failures=0; required_skips=(); run_postgres=true
POSTGRES_CONTAINER_NAME=test; POSTGRES_HOST_PORT=5433
unset SYSKNIFE_TEST_POSTGRES_URL
have() { return 0; }
podman() { printf 'podman %s\\n' "$*" >&2; return 1; }
docker() { printf 'docker was selected\\n' >&2; return 1; }
run_postgres_contract_group
'''
result = bash(code)
self.assertIn('podman run', result.stderr)
self.assertNotIn('docker was selected', result.stderr)


unittest.main(verbosity=2)
PY
Loading