chore: e2e-first testing; delete the unit suite, relocate release and contract guards

.amplifier/skills/amplifier-agent-bugfix/SKILL.md

@@ -132,9 +132,10 @@ amplifier-digital-twin check-readiness aa-e2e; echo "rc=$?"
 amplifier-digital-twin exec aa-e2e -- bash -lc '<the exact reported command>'
 ```
 
-Use `cli.py run --fresh` instead when you want the harness to build the box from
-scratch rather than reuse a warm one. Either way, run the reported command
-verbatim before running anything clever. If it does not reproduce, say so and
+Use `cli.py run <area>` instead of manual `up` + `exec` when the reported
+command is already covered by an E2E case: any `run` without `--skip-setup`
+provisions the same fresh box first, then runs the suite for you. Either way,
+run the reported command verbatim before running anything clever. If it does not reproduce, say so and
 find out what differs before going further; an unreproducible bug cannot be
 verified fixed.
 
@@ -340,12 +341,19 @@ that box does not exist, check.
 
 ```bash
 ls .amplifier/digital-twin-universe/profiles/
-grep -rn "install.sh\|install-amplifier-agent" tests/e2e/ | head -20
-```
-
-If a clean-install harness already exists, use it and treat this as class 1. If
-it exists but only probes reachability rather than running install and launch
-end to end, extending it is the smaller job. Either way, do not quietly proceed.
+cat .github/workflows/install-script.yml
+```
+
+A clean-install harness already exists: `.github/workflows/install-script.yml` runs the real
+`install.sh` (the pushed tag on a tag push, otherwise the latest release) in a bare
+`python:3.12-slim` container, then asserts the prepared-bundle cache was actually primed
+(`install.sh` swallows priming failures, so a bare `--help` check would miss that). It does
+not run as part of the E2E DTU harness, so grepping `tests/e2e/` for it will not find it; the
+`framework/provisioning/install-amplifier-agent.sh` script matched by that grep is a
+different thing, the E2E harness's own in-DTU install step, not a clean-install regression
+box. Treat this as class 1 and use `install-script.yml` (or extend it) rather than assuming
+no harness exists. If it exists but only probes reachability rather than running install and
+launch end to end, extending it is the smaller job. Either way, do not quietly proceed.
 STOP and put the choice to the user:
 
 ```
@@ -482,13 +490,10 @@ echo "launched pid $!"
 Re-launch the SAME subset to the SAME log name across iterations so runs are
 comparable. Only change the log name when the scope changes.
 
-Keep the fast local gates green as you go:
+Keep the fast local gate green as you go:
 
 ```bash
-uv run ruff check src/ tests/
-uv run ruff format --check src/ tests/
-uv run pyright src/
-uv run pytest tests/ -q
+make check
 ```
 
 Delegation works for the implementation loop. Hand a builder agent the pinned
@@ -507,18 +512,20 @@ ladder at rung 2 with the suite nearest the change.
 1. -k "<the regression case>"             the fix works
 2. cli.py run <area> --skip-setup         the suite it lives in
 3. cli.py run <area> <adjacent>           the neighbors it could have broken
-4. uv run pytest tests/ -m "not dtu"      everything non-DTU
-5. cli.py run                             all e2e suites
-6. cli.py run --fresh                     clean-box confirmation
+4. make check                             lint + types stay clean
+5. cli.py run                             all e2e suites, freshly provisioned:
+                                           clean-box confirmation
 ```
 
-Do not run `--fresh` while iterating. Reuse the warm named DTU with
-`--skip-setup` until the narrow scope is green.
+Do not run the full unscoped suite while iterating. Reuse the warm named DTU
+with `--skip-setup` until the narrow scope is green.
 
-Record known-red tests to a baseline before you widen, so only deltas count:
+Record known-red suites to a baseline before you widen, so only deltas count. There is no
+non-DTU test tier to fall back on for this; the baseline is the e2e run itself:
 
 ```bash
-uv run pytest tests/ -q -m "not dtu" 2>&1 | tail -40 > /tmp/baseline.txt
+uv run python tests/e2e/framework/cli.py run <area> <adjacent> --skip-setup -rxX \
+  2>&1 | tail -40 > /tmp/baseline.txt
 ```
 
 Name up front which existing failures are expected, so a known-bad suite is not
@@ -537,8 +544,8 @@ The Phase 3 classification is recorded, with the destination it implied
 The regression test fails on the old code and passes on the new
 If this was an evaluation concern: task id, trial count, and the scores before
   and after, from the same task at the same trial count
-Scoped suite green, plus adjacent suites, plus one --fresh run
-ruff check, ruff format --check, pyright src/, pytest tests/ -q all clean
+Scoped suite green, plus adjacent suites, plus one full run without --skip-setup
+make verify clean (lint, types, and every contract/release guard)
 Every hunk in the diff traces to the root cause. No unrelated refactors, no
   lockfile churn. If the fix was larger than the bug looked, the user agreed to it
 No pre-existing test was weakened or deleted
@@ -596,12 +603,15 @@ use a bracket regex the literal command line will not match:
 pkill -f "suites/skills[ ]-m[ ]dtu"
 ```
 
-**Orphaned tmux inside the DTU** after a killed run:
+**Orphaned tmux inside a DTU** after a killed eval run: the eval harness's DTU
+image carries tmux, so this cleans it up there.
 
 ```bash
-amplifier-digital-twin exec aa-e2e -- tmux kill-server
+amplifier-digital-twin exec aa-eval -- tmux kill-server
 ```
 
+The `aa-e2e` image has no tmux; this does not apply to the e2e harness.
+
 **Stale harness state.** `/tmp/amplifier-*-e2e/state.json` can point at a DTU
 that no longer exists, and the failure looks like a harness bug. If DTU
 discovery behaves strangely, delete it and let it re-provision.
@@ -625,7 +635,8 @@ uv run pytest ... -o addopts=""
 a piped run so a truncated tail is distinguishable from a killed process:
 
 ```bash
-timeout 115 uv run pytest tests/ -q 2>&1 | tail -20; echo "EXIT_CHAIN_DONE"
+timeout 115 uv run python tests/e2e/framework/cli.py run <area> --skip-setup \
+  2>&1 | tail -20; echo "EXIT_CHAIN_DONE"
 ```
 
 **DTU naming.** `aa-e2e` belongs to the E2E harness. A throwaway box for one

.amplifier/skills/amplifier-agent-finish-release-process/SKILL.md

@@ -272,8 +272,14 @@ sequencing, and the downstream call.
 npm and does not un-run the workflow. The recovery for a bad release is a new
 patch version, never a re-pointed tag.
 
-**The publish workflows run no tests.** The gate was Phase 4 of the start
-skill, before the PR. There is no second chance here.
+**The publish workflows are not blind to the gate anymore, but they still do not run e2e.**
+`ci.yml` now triggers on `v*` / `wrapper-v*` / `py-v*` tag pushes and runs the same `make
+verify` targets (lint, types, codegen/version/wheel/parity/wrapper guards) that gated the PR,
+and `publish-python.yml` runs `scripts/verify-wheel.py` immediately before the PyPI upload. So
+a tag push is not entirely unguarded. What still does not run in CI, on a tag or otherwise, is
+`tests/e2e/` or the evaluation suite: both need a DTU, which these runners do not have. The
+gate for those was Phase 4 of the start skill, before the PR, and there is no second chance
+for them here.
 
 **`workflow_dispatch` on `publish-python.yml` skips the version check.** It
 exists for recovery. Do not reach for it as a normal path, and never as a way

.amplifier/skills/amplifier-agent-new-feature/SKILL.md

@@ -198,12 +198,12 @@ subset and widen only after the narrow scope is green:
 2. -k "<a or b or c>"                     the whole red set for this feature
 3. cli.py run <feature>                   the whole suite
 4. cli.py run <feature> <adjacent>        regression check on neighbors
-5. cli.py run                             everything
-6. cli.py run --fresh                     clean-box confirmation
+5. cli.py run                             everything, freshly provisioned:
+                                           clean-box confirmation
 ```
 
-Do not run all suites while iterating. Do not run `--fresh` while iterating.
-Reuse the warm named DTU with `--skip-setup`.
+Do not run all suites while iterating. Reuse the warm named DTU with
+`--skip-setup`.
 
 ```bash
 rm -f /tmp/green_run.log && setsid bash -c \
@@ -219,13 +219,10 @@ expected and acceptable, so a known-bad suite is not mistaken for a regression.
 
 If a test looks wrong, escalate. Do not edit it.
 
-Also keep the fast local gates green as you go:
+Also keep the fast local gate green as you go:
 
 ```bash
-uv run ruff check src/ tests/
-uv run ruff format --check src/ tests/
-uv run pyright src/
-uv run pytest tests/ -q
+make check
 ```
 
 Delegation works well for the implementation loop. Hand a builder agent the plan
@@ -256,8 +253,8 @@ Wait for their verdict before calling the feature done.
 Confirm each of these, with evidence, not assertion:
 
 ```
-Scoped E2E suite green, plus adjacent suites, plus one --fresh run
-ruff check, ruff format --check, pyright src/, pytest tests/ -q all clean
+Scoped E2E suite green, plus adjacent suites, plus one full run without --skip-setup
+make verify clean (lint, types, and every contract/release guard)
 docs/spec/*.md updated in this same change
 No hardcoded paths. Would this work for someone else who checks out the repo?
 No secrets in code, tests, fixtures, logs, or committed config
@@ -308,12 +305,15 @@ use a bracket regex the literal command line will not match:
 pkill -f "suites/skills[ ]-m[ ]dtu"
 ```
 
-**Orphaned tmux inside the DTU** after a killed run:
+**Orphaned tmux inside a DTU** after a killed eval run: the eval harness's DTU
+image carries tmux, so this cleans it up there.
 
 ```bash
-amplifier-digital-twin exec aa-e2e -- tmux kill-server
+amplifier-digital-twin exec aa-eval -- tmux kill-server
 ```
 
+The `aa-e2e` image has no tmux; this does not apply to this harness.
+
 **Repo pytest addopts interfering** with a run:
 
 ```bash
@@ -325,7 +325,8 @@ uv run pytest ... -o addopts=""
 a piped run so a truncated tail is distinguishable from a killed process:
 
 ```bash
-timeout 115 uv run pytest tests/bundle -q 2>&1 | tail -20; echo "EXIT_CHAIN_DONE"
+timeout 115 uv run python tests/e2e/framework/cli.py run <feature> --skip-setup \
+  2>&1 | tail -20; echo "EXIT_CHAIN_DONE"
 ```
 
 **DTU naming.** `aa-e2e` belongs to this harness. `aa-eval` belongs to the eval

.amplifier/skills/amplifier-agent-start-release-process/SKILL.md

@@ -111,14 +111,18 @@ content. The durable artifact is `docs/spec/`. Hunt for:
 
 ```
 plan / PLAN / *-plan.md / implementation-plan.md / phase-*.md
-.ai_working/  scratch/  notes/  tmp/  WIP*
+.ai_working/  scratch/  tmp/  WIP*
 runs/            eval output: provider keys, full prompts, host paths
 *.log  *.tmp  *.bak  *.orig  *.rej  .DS_Store
 ```
 
 If a plan file exists and the user still wants it, it moves outside the repo,
 into `.ai_working/` or wherever they keep working notes. It does not ship.
 
+`notes/` is durable and must NOT be swept. It holds intentional, checked-in
+artifacts (e.g. coverage gaps, reproducibility notes), not scratch. See the
+`notes/` row in AGENTS.md's "What lives where" table.
+
 ### Comments and code that leak the process
 
 The tell is a comment that only makes sense to someone who watched the work
@@ -210,8 +214,9 @@ Do not touch the root `package.json`. It is `amplifier-agent-client-ts`, the
 pnpm workspace root manifest, and no workflow publishes it.
 
 If `PROTOCOL_VERSION` moved, verify both wrappers' pinned `--protocol-version`
-values, the `wrappers/conformance/` fixtures, `test_protocol_version_bump.py`,
-and the protocol version stated in `README.md` all agree.
+values, the `wrappers/conformance/` fixtures, and the protocol version stated
+in `README.md` all agree. `scripts/verify-versions.py` (`make verify-versions`)
+checks this cross-manifest consistency for you.
 
 ---
 
@@ -251,27 +256,31 @@ Confirm today's date rather than assuming it.
 
 ## Phase 4: Gate
 
-Nothing downstream will catch what you miss here. The publish workflows build
-and upload without running the test suite, so this local run is the gate.
+This local run is still most of the gate. `make verify` covers lint, types, and
+every contract/release guard (codegen staleness, cross-manifest versions, wheel
+contents, wrapper parity), and `ci.yml` now runs those same targets on every
+tag push too, so a broken `make verify` will also fail in CI after the tag is
+pushed, not just silently upload. `publish-python.yml` additionally runs
+`scripts/verify-wheel.py` right before the PyPI upload. What none of this
+covers is `tests/e2e/`: e2e and evals need a DTU, which CI does not have, so
+they never run there. This local run (plus the human loop from feature/bugfix
+work) is still the only gate for those.
 
 Always:
 
 ```bash
-uv run ruff check src/ tests/
-uv run ruff format --check src/ tests/
-uv run pyright src/
-timeout 115 uv run pytest tests/ -q 2>&1 | tail -20; echo "EXIT_CHAIN_DONE"
+make verify
 ```
 
-If wrappers or the protocol changed, also:
-
-```bash
-cd wrappers/typescript  && bun install && bun run build && bun run test
-cd wrappers/conformance && pnpm install && pnpm test
-```
+If wrappers or the protocol changed, `make verify` already covers
+`verify-parity` and `verify-wrapper` (TypeScript build + test), so there is
+nothing extra to run manually.
 
-Run these from the right directory. There is no aggregator script, and running
-engine tests from a wrapper directory produces a confusing pass.
+Run `make` targets from the repo root. `make verify-wrapper` cds into
+`wrappers/typescript/`. `make verify-parity` runs from the repo root; it only
+cds into `wrappers/conformance/` for the conditional `pnpm install` when
+`node_modules/` is missing. Either way there is no "run engine tests from a
+wrapper directory" mistake to make.
 
 Everything must be green before a PR opens. If something fails, fix it or stop
 and report it. Do not open a release PR on a red tree and plan to fix it in the

.github/workflows/ci.yml

@@ -1,18 +1,39 @@
 name: CI
 
+# The gate for every change AND every release.
+#
+# This repo is spec + e2e + eval driven. There is no unit test tier, so CI's job
+# is lint, types, and the standalone contract/release guards -- the same ones a
+# developer runs locally via `make verify`. Steps invoke Makefile targets rather
+# than open-coding commands so CI and local development cannot drift.
+#
+# NOT RUN HERE: `pytest tests/`. tests/ contains only tests/e2e/, which requires a
+# Digital Twin Universe (Incus + docker + amplifier-digital-twin). GitHub-hosted
+# runners cannot provide that, so every e2e test self-skips -- a green gate that
+# verifies nothing. Run the contract suite with `make e2e` on a machine with a DTU.
+# Do not add a pytest step here.
+
 on:
   push:
     branches: [main]
+    # Tags gate the release path: a tag push publishes to PyPI/npm, so it must
+    # clear the same gate as a PR. Without this, `git push --tags` shipped with
+    # zero verification.
+    tags:
+      - 'v*'
+      - 'py-v*'
+      - 'wrapper-v*'
   pull_request:
 
 # Cancel in-flight runs for the same ref when a new commit is pushed.
+# Never cancel a tag run -- it is the gate in front of a publish.
 concurrency:
   group: ci-${{ github.workflow }}-${{ github.ref }}
-  cancel-in-progress: true
+  cancel-in-progress: ${{ github.ref_type != 'tag' }}
 
 jobs:
-  python-tests:
-    name: Python tests + lint + types
+  gate:
+    name: Lint, types, and contract gates
     runs-on: ubuntu-latest
     timeout-minutes: 15
     steps:
@@ -30,10 +51,10 @@ jobs:
       - name: Install dependencies
         run: uv sync --all-extras --dev
 
-      # Node + pnpm are required by tests/test_conformance_parity.py, which shells
-      # out to `pnpm exec tsx runner_ts.ts` to cross-validate the TS runner against
-      # the Python runner. Without these, the conformance parity tests fail with
-      # FileNotFoundError: 'pnpm'.
+      # Node + pnpm are required by wrappers/conformance/verify-parity.py, which
+      # shells out to `pnpm exec tsx runner_ts.ts` to cross-validate the TypeScript
+      # runner against the Python runner. This is the ONLY cross-language check.
+      # Without these, `make verify-parity` fails with FileNotFoundError: 'pnpm'.
       - name: Set up Node.js
         uses: actions/setup-node@v4
         with:
@@ -44,19 +65,28 @@ jobs:
         with:
           version: 9
 
-      - name: Install conformance TS deps (for parity tests)
+      - name: Install conformance TS deps (for parity)
         working-directory: wrappers/conformance
         run: pnpm install --frozen-lockfile || pnpm install
 
-      - name: ruff (lint)
-        run: uv run ruff check src/ tests/
+      - name: make check (ruff lint + format + pyright)
+        run: make check
+
+      - name: make verify-codegen (protocol spec + schemas match the generator)
+        run: make verify-codegen
+
+      - name: make verify-versions (protocol version agrees everywhere)
+        run: make verify-versions
 
-      - name: pyright (types)
-        run: uv run pyright src/
+      - name: make verify-wheel (built wheel ships spec, schemas, fixtures, bundle)
+        run: make verify-wheel
 
-      - name: pytest
-        run: uv run pytest tests/ -q --tb=short
+      - name: make verify-parity (Python <-> TypeScript wire parity)
+        run: make verify-parity
 
+  # This is `make verify-wrapper`, the last piece of `make verify`. It is a
+  # separate job because it needs Bun rather than uv/pnpm, and because it is the
+  # gate in front of the npm publish in publish-wrapper.yml.
   ts-wrapper-tests:
     name: TypeScript wrapper tests
     runs-on: ubuntu-latest