ci(host-proofs): run the binary-gated real-host install proofs on every PR with pinned CLIs (#395)

* ci(host-proofs): run the binary-gated real-host install proofs on every PR with pinned CLIs

The host-install, packed host-install, and packed Claude plugin-validation
proofs skip their Claude/Codex legs when the CLIs are absent, which let #364
break both proofs on main unnoticed (#367/#368 repaired them from local runs).

- Pin @anthropic-ai/claude-code@2.1.250 and @openai/codex@0.147.0 as `hostCli`
  in each adapter's schema PROVENANCE.json; scripts/host-cli-pins.mjs reads
  the pins, refuses a pin that differs from observedCliVersion, installs them
  (re-running Claude's postinstall when npm blocks it), and fails closed with
  one diagnostic line when the CLI on PATH is not the pin.
- Add the host-install-proofs CI job (cached CLI prefix, version gate, build,
  test:host-install, test:host-install:packed, test:packed:native) honoring
  the docs-only skip. No secrets or login are needed.
- Harden the Codex interface proof: one shared pinned snapshot for both proof
  suites, installed manifest must equal the built artifact, validate against
  the pinned plugin schema, and reject fields outside the adapter's exported
  codexInterfaceFields.
- Document the job and the local commands in docs/local-ci.md and README.

* docs(local-ci): list dev-live-host among the CI-run host proofs

* fix(ci): key host CLI cache on package names and resolve npm global bin via prefix

Address the automated review on #395: the actions/cache key now includes each
pinned hostCli.package as well as its version, so a re-pin to a different
package with the same version misses the cache instead of reusing stale
binaries; and the installer derives npm's global bin directory from
`npm prefix -g` (<prefix>/bin on POSIX, the prefix itself on Windows)
instead of walking up from `npm root -g`.

* fix(ci): hash exact host CLI pins into the cache key

Sanitising package names for the actions/cache key can map distinct
packages (`@foo/bar`, `foo-bar`) to the same text; append a 16-hex SHA-256
of the exact `package@version` pairs so any re-pin misses the cache.

* fix(ci): probe installed host CLIs from the prefix only and reject bare --prefix

The post-install probe now runs with PATH limited to the prefix bin
directory plus the running Node, so a pre-existing claude/codex elsewhere
on the inherited PATH cannot mask an incomplete prefix. `install --prefix`
without a directory operand is rejected instead of silently installing
into the default global npm prefix.

Author: ScriptedAlchemy

.changeset/ci-host-install-proofs.md

@@ -0,0 +1,13 @@
+---
+"agent-bundle": patch
+---
+
+Pin the Claude Code and Codex CLI versions the real-host install proofs run
+against beside each adapter's schema provenance (`hostCli` in
+`src/adapters/schemas/{claude,codex}/PROVENANCE.json`, kept equal to
+`observedCliVersion`), and export the Codex adapter's declared
+`codexInterfaceFields` so the host-install proofs can reject an undeclared
+`interface` emission before comparing against their single pinned snapshot.
+Repository CI now installs the pinned CLIs and runs the host-install, packed
+host-install, and packed Claude plugin-validation proofs on every change,
+signed out and without secrets. No runtime behavior changes.

.github/workflows/ci.yml

@@ -175,6 +175,66 @@ jobs:
       # the nightly packed-matrix job and in pre-publish `pnpm check:release`.
       - run: pnpm check:release:ci
 
+  # Binary-gated real-host install proofs. The proof suites skip their Claude
+  # and Codex legs when the CLIs are absent, which is how #364 changed the
+  # Codex `interface.logo` emission and broke both proofs on main without CI
+  # noticing (#367/#368 repaired them from local runs). This job installs the
+  # exact CLI versions pinned in each adapter's schema PROVENANCE.json
+  # (`hostCli`, kept equal to `observedCliVersion` by scripts/host-cli-pins.mjs),
+  # fails closed if `claude`/`codex --version` differs from the pin, and runs
+  # the source-built proofs, the packed-tarball proofs, and the packed Claude
+  # plugin validation. Nothing here needs a login: every proof runs against an
+  # isolated HOME / CLAUDE_CONFIG_DIR / CODEX_HOME and no secrets are passed.
+  # Signed-in smokes (`claude -p`, `codex exec`, the Eval harnesses) stay in
+  # the opt-in native-host-smoke workflow.
+  host-install-proofs:
+    needs: changes
+    if: >-
+      ${{ !cancelled() && github.event_name != 'schedule' &&
+      (github.event_name != 'pull_request' || needs.changes.outputs.docs_only != 'true') }}
+    name: Host install proofs (Node 22.19)
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    steps:
+      - uses: actions/checkout@v7
+      - uses: pnpm/setup@v2
+        with:
+          cache: true
+          install: false
+          runtime: node@22.19.0
+      - run: pnpm install --frozen-lockfile
+      - name: Read pinned host CLI versions
+        id: pins
+        run: node scripts/host-cli-pins.mjs print
+      # The global npm prefix holding both CLIs (Claude Code and Codex ship
+      # platform-native binaries), keyed by OS, arch, and the exact pins
+      # (package name and version per host) so a deliberate re-pin of either
+      # misses the cache and installs fresh.
+      - name: Restore pinned host CLIs
+        id: host-cli-cache
+        uses: actions/cache@v6
+        with:
+          path: ${{ runner.temp }}/host-cli
+          key: host-cli-${{ runner.os }}-${{ runner.arch }}-${{ steps.pins.outputs.pins }}
+      - name: Install pinned host CLIs
+        if: steps.host-cli-cache.outputs.cache-hit != 'true'
+        run: node scripts/host-cli-pins.mjs install --prefix "$RUNNER_TEMP/host-cli"
+      # ubuntu runner: npm's global executables live in <prefix>/bin (the
+      # script's globalBinDirectory rule; Windows would be the prefix itself).
+      - name: Expose host CLIs on PATH
+        run: echo "$RUNNER_TEMP/host-cli/bin" >> "$GITHUB_PATH"
+      - name: Verify host CLI versions match the schema pins
+        run: node scripts/host-cli-pins.mjs verify
+      - run: pnpm build
+      - name: Host install proofs (source-built bundle)
+        run: pnpm test:host-install
+      - name: Host install proofs (packed tarball)
+        run: pnpm test:host-install:packed
+      - name: Packed Claude plugin validation proof
+        env:
+          AGENT_BUNDLE_PACKAGE_PREBUILT: '1'
+        run: pnpm test:packed:native
+
   # Release-boundary scaffolder template matrix (mcp-server, cli-tool) plus
   # the full packed pool — the nightly form of pre-publish `check:release`.
   packed-matrix:
@@ -207,7 +267,8 @@ jobs:
   rsc-runtime-micro-eval:
     # Deterministic end-to-end spot-check of the built RSC runtime artifacts
     # (hook -> RSC worker -> shared kernel state -> MCP tool lowering) without
-    # any real Claude/Codex host. Real native-host smokes stay skip-gated in
+    # any real Claude/Codex host. Login-free real-host install proofs run in
+    # the host-install-proofs job; signed-in native smokes stay skip-gated in
     # the manually dispatched native-host-smoke workflow on purpose.
     needs: changes
     if: >-

.github/workflows/native-host-smoke.yml

@@ -20,6 +20,10 @@ concurrency:
 jobs:
   native-host-smoke:
     # This runs only on a trusted runner with existing CLI login state; no workflow secrets are passed.
+    # The login-free subset (host install proofs, packed host install proofs, packed Claude plugin
+    # validation) also runs on every PR and main push in ci.yml's host-install-proofs job against the
+    # CLI versions pinned in the adapter schema PROVENANCE files; this workflow adds the signed-in
+    # `claude -p` / `codex exec` evidence on top, against whatever CLI the trusted runner has.
     runs-on: self-hosted
     strategy:
       fail-fast: false

docs/local-ci.md

@@ -21,8 +21,8 @@ Verify-equivalent leg on whatever Node is currently active, with the repo's
 normal local worker derivation. It skips the Node matrix and the
 examples/release/micro-eval gates, so it is a fast signal, not a merge gate.
 
-Docs-only PRs skip the hosted Verify, examples, release-gates, and micro-eval
-jobs. Docs-only means changes under `docs/` or `agent-patterns/`, changeset
+Docs-only PRs skip the hosted Verify, examples, release-gates, micro-eval, and
+host-install-proofs jobs. Docs-only means changes under `docs/` or `agent-patterns/`, changeset
 markdown (`.changeset/*.md`), or top-level markdown. Nested markdown elsewhere
 is treated as code. Pushes to `main` never use this skip. The allowlist and
 fail-open listing checks are implemented by `scripts/classify-docs-only.mjs`
@@ -75,6 +75,66 @@ duration × test census) is printed and written to
 `.worktrees/local-ci/summary.md` (plus `summary.json`); per-step logs land in
 `.worktrees/local-ci/logs/`. The command exits non-zero if any step fails.
 
+## Real-host install proofs
+
+The host-install proof suites (`host-install-proof.test.ts`,
+`packed-host-install-proof.test.ts`, `dev-host-install.test.ts`,
+`dev-live-host.test.ts`, and the packed Claude plugin validation in
+`packed-native-smoke.test.ts`) skip their
+Claude and Codex legs whenever the `claude` or `codex` CLI is not on PATH.
+That skip is what let #364 change the Codex `interface.logo` emission and
+break both proofs on `main` without CI noticing. Hosted CI therefore runs a
+dedicated `host-install-proofs` job (Node 22.19) on every PR and `main` push:
+
+1. `node scripts/host-cli-pins.mjs print` reads the pins — the `hostCli`
+   block in `packages/agent-bundle/src/adapters/schemas/claude/PROVENANCE.json`
+   (`@anthropic-ai/claude-code`) and `.../schemas/codex/PROVENANCE.json`
+   (`@openai/codex`). Each pin must equal that file's `observedCliVersion`,
+   so bumping the CLI CI runs is the same deliberate edit as re-pinning the
+   schemas observed against it.
+2. `node scripts/host-cli-pins.mjs install --prefix "$RUNNER_TEMP/host-cli"`
+   installs exactly those versions into a global npm prefix (cached by OS,
+   architecture, and pin), then re-runs Claude Code's `install.cjs` when npm
+   blocked its postinstall, which is what links the native binary.
+3. `node scripts/host-cli-pins.mjs verify` fails closed with one diagnostic
+   line per host when `claude --version` or `codex --version` on PATH is not
+   the pin.
+4. `pnpm build`, `pnpm test:host-install`, `pnpm test:host-install:packed`,
+   and `AGENT_BUNDLE_PACKAGE_PREBUILT=1 pnpm test:packed:native`.
+
+None of it needs a login or a secret: every proof runs against an isolated
+`HOME` / `CLAUDE_CONFIG_DIR` / `CODEX_HOME`, and the subcommands involved
+(`claude plugin marketplace add`, `plugin install`, `plugin list --json`,
+`plugin details`, `plugin validate --strict`; `codex plugin marketplace add`,
+`plugin add`, `plugin list`) work signed out. The job passes no workflow
+secrets.
+
+To run the same proofs locally:
+
+```sh
+pnpm check:host-cli           # your PATH claude/codex must match the pins
+pnpm build
+pnpm test:host-install
+pnpm test:host-install:packed
+AGENT_BUNDLE_PACKAGE_PREBUILT=1 pnpm test:packed:native
+```
+
+If your machine tracks a newer CLI, install the pinned versions into a
+throwaway prefix and put it first on PATH for the run:
+
+```sh
+node scripts/host-cli-pins.mjs install --prefix /tmp/host-cli
+PATH=/tmp/host-cli/bin:$PATH pnpm check:host-cli
+PATH=/tmp/host-cli/bin:$PATH pnpm test:host-install
+```
+
+What stays binary-gated *and* login-gated, and therefore outside this job:
+the `claude -p` session proofs (`AGENT_BUNDLE_HOST_INSTALL_CLAUDE_SESSION`,
+`AGENT_BUNDLE_NATIVE_CLAUDE_SMOKE`), the Codex `exec` smoke
+(`AGENT_BUNDLE_NATIVE_CODEX_SMOKE`), and the packed Eval smokes
+(`AGENT_BUNDLE_PACKED_NATIVE_{CLAUDE,CODEX}_SMOKE`). Those still run only in
+the opt-in `native-host-smoke` workflow on a signed-in runner.
+
 ## Node provisioning
 
 The runner introduces no new tooling. For each hosted runtime line
@@ -116,6 +176,11 @@ never lets it drop below what its own pool shape requires.
   hosted-only, PR-time check.
 - **package-preview** (pkg.pr.new) and the **release publish** workflow are
   publish-side effects, not checks; nothing about them gates a merge.
+- **host-install-proofs** needs the pinned `claude` and `codex` CLIs on PATH
+  (see [Real-host install proofs](#real-host-install-proofs)). The local gate
+  does not install host CLIs into its legs, so run those proofs by hand with
+  the commands above when a change touches adapter emission, the installers,
+  or the proof suites.
 - **native-host-smoke** needs signed-in Claude/Codex CLIs and is opt-in even
   on hosted CI.
 - **Environment skew**: hosted runners are `ubuntu-latest`, and hosted jobs

package.json

@@ -25,6 +25,7 @@
     "typecheck": "tsc --noEmit && tsc --project packages/workbench/tsconfig.json && tsc --project packages/create-agent-bundle/tsconfig.json",
     "check": "pnpm build && pnpm test:unit && pnpm test:route-unit && pnpm test:projection && pnpm test:integration:run && pnpm lint && pnpm typecheck",
     "check:local-ci": "node scripts/local-ci.mjs",
+    "check:host-cli": "node scripts/host-cli-pins.mjs verify",
     "docs:runtime-topology": "node scripts/rsc-runtime-topology.mjs --root . --output docs/architecture/rsc-runtime-workbench.md",
     "eval:spot": "pnpm build && pnpm --filter @agent-bundle/rsc-agent-runtime-demo build && pnpm --filter @agent-bundle/rsc-agent-runtime-demo exec rstest run tests/micro-eval.spot.test.ts --config rstest.config.ts",
     "check:runtime-topology": "node scripts/rsc-runtime-topology.mjs --root . --output docs/architecture/rsc-runtime-workbench.md --check",

packages/agent-bundle/README.md

@@ -437,7 +437,12 @@ prove native-host install or dispatch, or an install mode that copies the
 artifact elsewhere. `host-install` is separate installed-layout process
 evidence: its deterministic adapter-simulator lane is unconditional, available
 Claude and Codex binaries also prove their public install paths, and Cursor
-records its unavailable non-interactive host-session surface explicitly.
+records its unavailable non-interactive host-session surface explicitly. The
+Claude and Codex legs skip when those binaries are absent, so the repository's
+CI runs them on every change against the exact CLI versions pinned beside each
+adapter's schema provenance (`hostCli` in `src/adapters/schemas/*/PROVENANCE.json`),
+signed out and with no secrets; only the `claude -p` session and Eval smokes
+remain login-gated.
 
 ### Contract matrix (`runContractMatrix` / `runPackedContractMatrix` / `runInstalledHostContractMatrix`)
 

packages/agent-bundle/src/adapters/codex.ts

@@ -241,7 +241,12 @@ const isAbsoluteUrl = (value: unknown): value is string => {
   }
 };
 
-const codexInterfaceFields = Object.freeze([
+/**
+ * Every `interface` field the adapter can emit — generated or authored through
+ * the `codex.interface` extension. The host-install proofs check installed
+ * manifests against this declared set so an undeclared emission cannot land.
+ */
+export const codexInterfaceFields = Object.freeze([
   'brandColor',
   'capabilities',
   'category',