Skip to content

feat(runner): add nself runner provision/verify for CI runner hosts (G-012) - #411

Merged
acamarata merged 3 commits into
mainfrom
p6/g012-runner-provisioning
Sep 12, 2026
Merged

acamarata merged 3 commits into
mainfrom
p6/g012-runner-provisioning

Conversation

@acamarata

Copy link
Copy Markdown
Collaborator

Summary

Closes G-012. CI runner hosts were hand-built, so required system dependencies were discovered only when a job failed mid-run, and two hosts advertising the same GitHub Actions labels (self-hosted,Linux,X64) could silently drift apart depending on which one had gh/zip/unzip installed. Separately, a playwright install --with-deps failure (sudo needing a terminal) fell back to a Chromium install with no system libs, which surfaced hours later as a misleading generic Playwright error instead of the real libnspr4.so cause.

  • internal/runner/manifest.yaml (embedded): the single declarative dependency set — git, jq, yq, gh, make, zip, unzip, curl, docker, build-essential, pkg-config, libssl-dev, and the full Playwright/Chromium runtime lib list — with a reason on every entry tying it back to the incident that added it.
  • internal/runner.Executor: local or SSH (reusing internal/deploy's exported SSH primitives, no second SSH code path), so every provision/verify step is one command string a test can assert against without touching a host.
  • nself runner verify [--host ...]: runs the manifest against one or more hosts and renders a text (or --json) parity matrix, with explicit DriftFindings when the same check disagrees across hosts — the scenario that hid for hours on 2026-09-11. Includes a dedicated ldd-based check against any cached Chromium binary.
  • nself runner provision [--host ...]: idempotently installs the manifest's packages, creates the runner user with passwordless sudo, and registers N runner instances as systemd services via the runner's own svc.sh. Refuses outright (does not silently replace) if the shared _work directory is a symlink — git resolves symlinks before matching includeIf.gitdir:, so a symlinked _work silently breaks actions/checkout's credential injection (fatal: could not read Username for 'https://github.com'), which is what destroyed 63 jobs.
  • Registered runner in the command group table (advanced) and the error-harness coverage table; regenerated the CLI command inventory (SPORT F02 mirror, .github/wiki/Commands.md, new cmd-runner.md).

Kept compatible with the P7-E5-W4-S1-T4 "runner fleet v1" ticket, which is nself ci serve's own RoleCI/Prober dispatch across boxes running nSelf's built-in CI daemon — a different concern from provisioning/auditing the GitHub Actions self-hosted runner host itself. No naming or package overlap.

Test plan

  • go build ./... — clean
  • go vet ./... — clean
  • gofmt -l on every changed file — clean
  • golangci-lint run ./internal/runner/... ./cmd/commands/... — 0 issues
  • go test ./internal/runner/... ./cmd/commands/... — all pass (95.9% coverage on internal/runner); tests use a fake Executor recording command strings, never a real host or network call
  • make wiki-check — all 51 wiki command pages current, all links resolve
  • gitleaks scan — no findings in any new file
  • Reviewer: confirm --host/SSH flag naming reads well against the eventual P7-E5-W4-S1-T7 LAN hardware verification ticket

CI runner hosts were hand-built, so required system dependencies were
discovered only when a job failed mid-run, and two hosts advertising the
same GitHub Actions labels (self-hosted,Linux,X64) could silently drift
apart depending on which one had gh/zip/unzip installed. A Playwright
--with-deps sudo failure also produced a Chromium binary missing shared
libs, which then surfaced hours later as a misleading generic error.

Adds internal/runner: a declarative dependency manifest (manifest.yaml,
embedded) covering git/jq/yq/gh/make/zip/unzip/curl/docker/build tooling
plus the Playwright/Chromium runtime libs, an Executor seam (local or SSH,
reusing internal/deploy's SSH primitives) so every check/step is a plain
command string asserted in tests without touching a real host, a verify
path that runs the manifest against N hosts and renders a cross-host
parity matrix plus explicit drift findings, a dedicated ldd check against
any cached Chromium binary, and a provision path that installs packages,
creates the runner user with passwordless sudo, and registers runner
instances as systemd services — refusing outright if the shared _work
directory is a symlink, since git resolves symlinks before matching
includeIf.gitdir and that silently breaks actions/checkout's credential
injection.

Wires `nself runner` (provision, verify) into cmd/commands, adds it to the
command group table and error harness, and regenerates the CLI command
inventory (SPORT F02, .github/wiki/Commands.md, cmd-runner.md).
@acamarata
acamarata merged commit d46255d into main Sep 12, 2026
31 checks passed
@acamarata
acamarata deleted the p6/g012-runner-provisioning branch September 12, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant