feat(s03): additive migration for task requirements, rehearsed on a restored store

CHANGELOG.md

@@ -5,6 +5,28 @@ All notable changes to Teploy Ship are recorded here.
 ## [Unreleased]
 
 ### Added
+- **The S03 requirements store has its migration, rehearsed on the real
+  store's backup (008).** `ship_task_requirements` — the record of which
+  statements were ACCEPTED as a task's requirements and which were later
+  waived, by whom and why (`src/task-requirements.ts`, drafted in an earlier
+  slice) — now has migration `008-ship-task-requirements` in `src/migrations.ts`,
+  the same additive new-table convention as 006/007: ledger-recorded on every
+  store, guarded by the write-shaped shape probe and the DDL-parity test, and a
+  rename-aside rebuild for a shape that has in fact never been released (so the
+  limb is unreachable today and nothing is ever copied or dropped). The store
+  itself stays deliberately UNWIRED — no route, worker path or UI writes it;
+  the `taskRecord()` projection is still read-only and a write path remains
+  future work gated on the programme's S03 rollout list. The migration was
+  rehearsed on a restored copy of the production store (verified
+  `pre-80c9187-shadow-deploy-2026-10-04` backup, isolated throwaway engine,
+  live containers untouched): additive-only across 45 tables / 4223 rows (only
+  `ship_migrations` moved, 7→8), replay a no-op, and the store's primary-key
+  and conditional-update semantics (idempotent add, conflict refusal,
+  exactly-once waiver) verified on the real engine rather than the unit-test
+  fake. Driver: `scripts/check-task-requirements-migration.mjs`
+  (`SHIP_ISOLATED_CHECK=1` + `NUCLEUS_URL`, the check-intake-concurrency
+  isolation contract); receipt:
+  `evals/receipts/2026-10-04-s03-requirements-migration.json`.
 - **The takeover panel has a BROWSER tab (S12's last surface).** While a
   lease is held, the operator drives a real headless Chromium running inside
   the sandbox — the same one the visual and flow rungs use (the node and go

NEXT_SESSION.md

@@ -16,7 +16,7 @@ Wave 4's state ("implemented and checked by automated tests only") is now partly
 
 1. **Shadow-log review** after soak (placement/policy/budget/tool-manifest/knowledge-provenance JSONL + reports); set `SHIP_MODEL_ROUTING_POLICY` so the routing shadow has something to record. Only then discuss any `on` flips.
 2. **S08 wiring** (approved): findings into PR body + webhook behind a default-off flag; worker-path change, so shadow-first per the standing rule.
-3. **Live proofs still open**: S01 credential proofs on a real sandbox with a private repo; S03 storage migration (write the additive migration, rehearse on a restored copy — backup `pre-80c9187-shadow-deploy-2026-10-04` is available and verified); S27 real teploy-adapter run against a scratch target; S19 doctor probes against the real store/clock; human observation of dashboard users.
+3. **Live proofs still open**: S01 credential proofs on a real sandbox with a private repo; S27 real teploy-adapter run against a scratch target; S19 doctor probes against the real store/clock; human observation of dashboard users. (S03's storage migration is done: migration 008 rehearsed on the restored `pre-80c9187` copy, receipt in `evals/receipts/2026-10-04-s03-requirements-migration.json`; the store stays unwired — a write path is separate future work.)
 4. **Code still unfinished**: S04 and S12 barely started; most of S07 beyond the grounding check; S10 follow-ups (offline/error states, other roles, screen readers); wiring the inert modules (S15/S17 into delivery+incidents, S19 snapshot producer + restore-check, S18 worker tree provisioning, S07 into the plan-park point).
 5. **Re-grade retained runs where possible**: only transcripts were preserved for pre-batch runs, so regrades are limited to transcript+fixture-verifiable scenarios; write supplementary `regrade-*.json` beside the records, never replacing originals.
 6. **Deferred**: configuration B of the S02 comparison (needs real API spend, owner-gated).

docs/SHIP_RELEASE_PROGRAMME_2026-09-21.md

@@ -483,14 +483,14 @@ Baseline reproduced in this session: `pnpm run lint` clean; root suite 1,412/1,4
 | S01 | Git credential placement audited; scrub now runs in the same shell as the credentialed clone/warm fetch | Yes | Sequencing test fails on previous code; failed-fetch probe is a regression pin only; env-mode real-git probe fails in argv mode (negative control) | No | `SHIP_GIT_CREDENTIAL=env` (off by default) keeps the token out of argv and config; proven on `LocalExecutor` with a real git and a local HTTP server, not through the Sandbox daemon. Needs: daemon `env` forwarding, sandbox git ≥ 2.31, a live private-repo proof on Forgejo and GitHub, then flipping the default (see `AUDIT_OPEN.md`). Preview egress is owned by teploy-cli, not this repo; no change made. Command-regex item `teploy-ship-04` unchanged |
 | S02 | Approval stops classified `authority-hold`, separate from `harness-error`; hash-linked reclassification of the migration run | Yes | Executor test fails on the old classification; manifest validation passes | No new model runs | Held-out set, clean current baseline, review-grader lexical limit, authorised-approval path for the migration scenario, n=3 repeats |
 | S28 | `runTiming(events)` derives first-step, human-wait, timer and active time with explicit unknowns | Yes, not yet wired into any export or UI | 7 unit tests including open waits and missing timestamps; not exercised against live logs | No | Wire into audit/JSON and UI; queue-claim event; OpenTelemetry export; review precision and rework measures |
-| S03 | `taskRecord()` projects execution / acceptance / delivery from existing records; `src/task-requirements.ts` is a drafted, unwired store for accepted and waived requirements (new table, no migration) | Yes, a read-only projection; not wired into a route or UI | 8 unit tests: finished ≠ accepted, revision supersedes approval, denied ≠ absent, delivery never inferred | No | Persisting requirements, acceptance criteria and waivers (needs an additive migration rehearsed on a restored copy); wiring the projection; versioned plans; lost-response / two-tab / steer scenarios on a real run |
+| S03 | `taskRecord()` projects execution / acceptance / delivery from existing records; `src/task-requirements.ts` is a drafted, unwired store for accepted and waived requirements (migration 008 added and rehearsed on a restored production copy, 2026-10-04) | Yes, a read-only projection; not wired into a route or UI; no write path enabled | 8 unit tests: finished ≠ accepted, revision supersedes approval, denied ≠ absent, delivery never inferred; migration unit tests + rehearsal receipt (`evals/receipts/2026-10-04-s03-requirements-migration.json`) | No | Wiring the projection; authorisation for accept/waive; versioned plans; lost-response / two-tab / steer scenarios on a real run |
 | S13 | Steering is refused (route) and not offered (UI) for runs whose harness cannot consume it, from one capability table | Yes, steering only | 4 unit tests on the table; web tests and build pass; the route refusal itself has no automated test | No | Full capability declaration (tools, browser, interruption, recovery, accounting); conformance journeys per harness; comparative results |
 | S05, S06, S08, S09 | Not started this session | Existing foundations only | | | Greenfield and unfamiliar-repo journeys (first-batch item 5) |
 | S18 | Not started this session | Existing API/client path only | | | Recorded two-repository integration-check kind (first-batch item 6) |
 | S18 | `integrationStatus()` in `src/integration-evidence.ts`: the contract an executed two-repository test's evidence must satisfy (exact producer/consumer revisions, static never satisfies executed, stale on upstream move, failed/not-run/unknown all block) | Yes, a pure contract; no executor and not wired into coordination | 8 unit tests | No | The executed integration-check kind itself (a non-scan journey with authority over a two-repo workspace and the real fixture run), wiring the contract into the coordination completion gate behind an opt-in, live producer/consumer run |
 | All others | Not started | Per plan table | | | As in the plan |
 
-First-executable-batch items: **(1)** mostly done — origin/main identity matches the plan, the baseline is reproduced and the [capability inventory](CAPABILITY_INVENTORY_2026-10-03.md) is written (subagent sweep, partly spot-checked); the S28 fields are only the derivation above. **(2)** partly done — hold classification fixed; deterministic preflight run (manifest valid, 12/12 dry-runs exit 0, grader negative controls 8/8, unauthorised spend refused with exit 2); the paid canary and batch proposal are not done (no spend authorised or attempted). **(3)** partly done — records mapped and projected (S03 row); no migration, no live lost-response/restart/steer rehearsal. **(4)** partly done — credential placement above; preview isolation is a teploy-cli change plus a live proof on the preview target, neither possible from here. **(5)** and **(6)** not started.
+First-executable-batch items: **(1)** mostly done — origin/main identity matches the plan, the baseline is reproduced and the [capability inventory](CAPABILITY_INVENTORY_2026-10-03.md) is written (subagent sweep, partly spot-checked); the S28 fields are only the derivation above. **(2)** partly done — hold classification fixed; deterministic preflight run (manifest valid, 12/12 dry-runs exit 0, grader negative controls 8/8, unauthorised spend refused with exit 2); the paid canary and batch proposal are not done (no spend authorised or attempted). **(3)** partly done — records mapped and projected (S03 row); migration 008 written and rehearsed on a restored production copy (2026-10-04); no live lost-response/restart/steer rehearsal. **(4)** partly done — credential placement above; preview isolation is a teploy-cli change plus a live proof on the preview target, neither possible from here. **(5)** and **(6)** not started.
 
 ### Execution status, wave 2 (2026-10-03, origin/main after PR #31)
 
@@ -574,9 +574,9 @@ Small specifications per the slice discipline above, recorded against their exis
 - **Evidence today:** `taskRecord()` can only show what each run's input said; nothing records which statements were accepted as requirements, or that one was waived and by whom (inventory, "Stored records").
 - **Intended behaviour:** a per-task requirement record. Adding is idempotent for identical content and a refused conflict for different content under the same id (no silent last-write-wins). A waiver needs an actor and a reason, applies once, keeps the requirement visible, and never becomes evidence that it was met. Who may accept or waive is the caller's policy, not the store's.
 - **Owning layer / contracts:** Ship (`src/task-requirements.ts`: interface plus memory, file and Nucleus stores). No Neutron or Nucleus change.
-- **Migration / recovery:** none required. The store creates its own new table (`ship_task_requirements`, primary key `req_key`) with `CREATE TABLE IF NOT EXISTS`, the same additive pattern as the other stores, so no existing run, parked fingerprint or populated table is altered. Revert is not wiring it; the table is inert and can stay.
-- **Independent acceptance (done):** 7 tests across all three stores: idempotent add, conflict refusal, concurrent-writer race on the primary key, waiver rules and exactly-once, per-task ordering, input limits, additive-only SQL. The Nucleus tests use a fake of my own writing, **not a real Nucleus**.
-- **Not done, and needed before any rollout:** rehearse on a restored production copy (`docs/UPGRADING.md`); a real-Nucleus run of the primary-key and conditional-update behaviour; wiring into enqueue (record the initial request and each follow-up as requirements) and the run page; authorisation for accept/waive on the existing approval grants; feeding waived/active requirements into `taskRecord()` acceptance; scenarios for two tabs, lost response and steering on a real run.
+- **Migration / recovery:** additive migration `008-ship-task-requirements` (src/migrations.ts, the 006/007 new-table convention: ledger-recorded, write-shaped shape probe, DDL-parity guard; rename-aside limb unreachable because no other shape was ever released). Rehearsed 2026-10-04 on a restored copy of the production store (backup `pre-80c9187-shadow-deploy-2026-10-04`): additive-only on 45 tables / 4223 rows (only ship_migrations moved, 7→8), replay a no-op, and the store's primary-key and conditional-update behaviour verified on the real engine. Receipt: `evals/receipts/2026-10-04-s03-requirements-migration.json`, driver `scripts/check-task-requirements-migration.mjs`. Revert is not wiring it; the table is inert and can stay.
+- **Independent acceptance (done):** 7 tests across all three stores: idempotent add, conflict refusal, concurrent-writer race on the primary key, waiver rules and exactly-once, per-task ordering, input limits, additive-only SQL. The Nucleus tests use a fake of my own writing, **plus the real-engine rehearsal above**.
+- **Not done, and needed before any rollout:** wiring into enqueue (record the initial request and each follow-up as requirements) and the run page; authorisation for accept/waive on the existing approval grants; feeding waived/active requirements into `taskRecord()` acceptance; scenarios for two tabs, lost response and steering on a real run.
 - **Evidence location:** `src/task-requirements.test.ts`.
 
 #### S01 — environment-supplied Git credential (implemented, off by default)

evals/receipts/2026-10-04-s03-requirements-migration.json

@@ -0,0 +1,63 @@
+{
+  "rehearsal": {
+    "date": "2026-10-04",
+    "purpose": "S03: prove migration 008-ship-task-requirements and the task-requirements store against a real Nucleus holding a restored copy of the production store",
+    "archive": "/deployments/ship/_backups/pre-80c9187-shadow-deploy-2026-10-04/nucleus-data-full.tgz",
+    "archiveSha256": "verified against its .sha256 sidecar (sha256sum -c) + gzip -t",
+    "engineImage": "ghcr.io/neutron-build/nucleus:v1.1.1 (the image of the live ship-nucleus container, read from docker ps -a)",
+    "isolation": "restore unpacked into /root/s03-rehearsal-s03104851 only; throwaway engine container ship-s03-proof-104851 on a private docker network (no published port); check script run from a node:22-bookworm container on the same network; only the proof container, network and directory were removed afterwards",
+    "liveContainersUntouched": "ship-web-80c9187, ship-worker-80c9187, ship-nucleus, ship-gateway-web-c0b07dd Up throughout (verified before and after; no restart)",
+    "engineBootMs": 3000,
+    "checkWallMs": 951,
+    "cleanup": "proof container + network + rehearsal dir removed; /deployments/ship and all ship-* containers never touched"
+  },
+  "check": {
+    "pass": true,
+    "scope": "migration 008 + task-requirements store on a restored copy; no production path",
+    "startedAt": "2026-10-04T09:10:35.393Z",
+    "finishedAt": "2026-10-04T09:10:35.564Z",
+    "url": "postgres://ship-s03-proof-104851:5432/nucleus",
+    "tablesSource": "SHOW TABLES",
+    "tablesPopulatedBefore": 37,
+    "totalRowsBefore": 4223,
+    "tablesBefore": {
+      "ship_akiroo_cursor": 2, "ship_artifact_expiry": 0, "ship_artifacts": 217,
+      "ship_attributed_spend": 89, "ship_bulletin_boards": 0, "ship_bulletin_posts": 0,
+      "ship_code_chunks": 329, "ship_code_files": 244, "ship_code_rates": 2,
+      "ship_code_repos": 6, "ship_connect_requests": 5, "ship_delivery": 7,
+      "ship_docs": 778, "ship_docs_001": 170, "ship_docs_004": 178,
+      "ship_evidence": 1, "ship_fleet": 1, "ship_fleet_capacity": 1,
+      "ship_fleet_load": 1, "ship_governance": 1, "ship_launch_chunks": 149,
+      "ship_launch_commits": 140, "ship_launch_dispositions": 0, "ship_launches": 140,
+      "ship_live": 0, "ship_memory": 205, "ship_memory_003": 13,
+      "ship_migrations": 7, "ship_outbox": 0, "ship_placement": 384,
+      "ship_policies": 1, "ship_projects": 18, "ship_projects_v2": 24,
+      "ship_repo_stats": 206, "ship_runtime_config": 531, "ship_spend": 253,
+      "ship_spend_holds": 16, "ship_steer": 5, "ship_steer_002": 1,
+      "ship_tasks": 58, "ship_tasks_005": 18, "ship_tasks_v2": 2,
+      "ship_unpriced_runs": 20, "ship_users": 0, "ship_workspace_content": 0
+    },
+    "tablesAfter": "identical to tablesBefore on every table except ship_migrations: 7 -> 8 (the 008 ledger row). ship_task_requirements was absent before the store-proof leg and is created by the store DDL, empty",
+    "migrate": {
+      "firstPassAppliedIds": [],
+      "firstPassNote": "ledger-only, the designed path: the table does not exist on the real store (the store is unwired), so needed() is false and 008 is recorded without table surgery. The rename-aside limb is covered by the unit tests against a stale shape.",
+      "firstPassMs": 34,
+      "ledgerIds": [
+        "001-ship-docs-source-ranon", "002-ship-steer-consumed-turn", "003-ship-memory-note-id",
+        "004-ship-docs-actor", "005-ship-tasks-requested-by", "006-ship-runtime-config",
+        "007-ship-connect-requests", "008-ship-task-requirements"
+      ],
+      "replayAppliedIds": [],
+      "replayMs": 12
+    },
+    "storeProof": {
+      "ms": 40,
+      "idempotentAdd": true,
+      "conflictRefused": true,
+      "waiverExactlyOnce": true,
+      "waivedStaysListed": true,
+      "rowsCreated": 2,
+      "cleanup": "proof rows deleted (final ship_task_requirements count 0, all other tables unchanged); the empty table and the 008 ledger row stay in the throwaway copy only"
+    }
+  }
+}