feat: release-validation report + readiness hard gate (M2)

Add the Heart-owned release-validation tier: a tracked validation_report.json
and an ingest-and-judge CLI that proves the exact source about to ship was
built, TestPyPI-published, wheel-installed, and exercised at release fidelity.

- heart/validate.py: schema_version 1 report + `pyauto-heart validate --ingest`.
  Folds the M1 TestPyPI rehearsal artifact, a {repo: sha} commit_shas.json, and
  (from M3) a wheel-based integration report into one report; computes
  release_ready (pass/fail axis) and persists it to state + a history archive.
  Ingest-and-judge ONLY: never dispatches a build, never talks to GitHub, never
  mutates a repo.
- heart/readiness.py: hard gate consuming the report. GREEN-for-release now
  requires a fresh passing report whose commit_shas match the current main HEADs
  (via ci_status.head_sha) under the `release` profile; absent/stale-by-age/
  SHA-mismatch/wrong-profile -> YELLOW; a failed stage -> RED.
- state.py aggregates validation_report.json; status.py renders it;
  bin/pyauto-heart gains the `validate` subcommand.
- health_agent/capabilities.yaml registers the `validate` capability +
  validation_report signal, the release_gate rule, and the Release/Health Agent
  boundary so the Brain agents surface it manifest-driven.
- docs/release_validation.md defines (not yet wired — M3) the `release` env
  profile and the wheel-install requirement as acceptance criteria.
- Tests: tests/test_validate.py (ingest) + readiness gate paths
  (absent/stale-by-SHA -> YELLOW, fresh-pass -> GREEN, fail -> RED). 166 pass.

Co-Authored-By: Claude <noreply@anthropic.com>

Author: claude

bin/pyauto-heart

@@ -24,6 +24,8 @@ SUBCOMMAND_ORDER=(
   readiness
   logs
   fix
+  "# Release validation (ingest-and-judge only; never dispatches a build)"
+  validate
   "# Deep checks (on-demand; too slow for the tick)"
   verify_install
   "# URL hygiene (monitoring only — does not gate releases)"
@@ -40,6 +42,7 @@ declare -A SHORT_DESC=(
   [tick]="One-shot refresh of the state cache"
   [status]="Print the cached state with green/yellow/red colouring"
   [readiness]="Print the release-readiness verdict (green/yellow/red + score)"
+  [validate]="Ingest release-validation artifacts into validation_report.json"
   [logs]="Tail the daemon log"
   [fix]="Bundle context for a specific topic and emit a Claude command"
   [verify_install]="Deep install-readiness: pip & conda install-path checks (slow)"
@@ -200,6 +203,35 @@ cmd_readiness() {
   exec env PYTHONPATH="$HEART_HOME" python3 -m heart.readiness "$@"
 }
 
+help_validate() { cat <<EOF
+pyauto-heart validate --ingest <artifacts...> [--profile P] [--testpypi-version V]
+                      [--commit-shas FILE] [--out FILE] [--json]
+
+Ingest-and-judge ONLY. Consume the release-validation artifacts the Brain
+Release Agent has already collected — the M1 TestPyPI rehearsal artifact
+(rehearsal.json / testpypi_version.txt), a {repo: sha} commit_shas.json, and
+(from M3) the wheel-based integration report.json — and fold them into a single
+\$HEART_STATE_DIR/validation_report.json (plus a validation_history/ archive).
+
+This NEVER dispatches a build, talks to GitHub, or mutates any repo. All
+dispatching/polling/downloading is the Brain Release Agent's job; Heart is spec
++ ingest + verdict, credential-free.
+
+The report carries release_ready (pass/fail axis), testpypi_version, profile,
+per-repo commit_shas, per-stage status, per-project counts, failures and run
+URLs. \`pyauto-heart readiness\` consumes it as a HARD gate: GREEN-for-release
+now requires a fresh passing report whose commit_shas match the current main
+HEADs under the 'release' profile; absent/stale/mismatch → YELLOW; a failed
+stage → RED.
+
+With no --ingest, prints the currently persisted validation_report.json.
+EOF
+}
+cmd_validate() {
+  mkdir -p "$HEART_STATE_DIR"
+  exec env PYTHONPATH="$HEART_HOME" python3 -m heart.validate "$@"
+}
+
 help_logs() { cat <<EOF
 pyauto-heart logs [--tail N] [--follow]
 

docs/release_validation.md

@@ -0,0 +1,98 @@
+# Release validation — the `release` profile and acceptance criteria
+
+This document is the **spec** half of Heart's release-validation tier. Heart
+owns the *definition* of what a release-grade validation run must do; it does not
+execute the build or the integration run (that is the Brain Release Agent
+dispatching Build's `release.yml` and the evolved `workspace-validation.yml`).
+
+M2 (this milestone) ships the report schema, `pyauto-heart validate --ingest`,
+and the readiness hard gate. The `release` env profile and the wheel-install
+requirement below are **defined here as acceptance criteria for M3** — they are
+not yet wired into `workspace-validation.yml`. Capturing them now keeps them from
+being lost between milestones.
+
+## Why a distinct profile
+
+The per-PR smoke gate and the release gate are different jobs:
+
+- **smoke** — a fast structural / integration check: *does the model compose and
+  the script run end-to-end?* Cheap, runs on every PR.
+- **release** — *does the exact source about to ship, installed from the built
+  wheel, pass at release fidelity?* Slow, runs only for a release rehearsal.
+
+Both tiers' `config/build/env_vars.yaml` today default to smoke values
+(`PYAUTO_TEST_MODE=2`, `PYAUTO_SMALL_DATASETS=1`, `PYAUTO_DISABLE_JAX=1`,
+`PYAUTO_FAST_PLOTS=1`). That is correct for a per-PR smoke and wrong for a
+release gate. The two must be **named, distinct profiles** so a release run
+cannot silently inherit smoke fidelity.
+
+## The `release` profile (acceptance criteria, wired in M3)
+
+The intended release-fidelity env is already documented in
+`PyAutoBuild/.github/workflows/release.yml`:
+
+| Tier                 | `PYAUTO_TEST_MODE`               | `PYAUTO_SMALL_DATASETS` | `PYAUTO_FAST_PLOTS` |
+|----------------------|----------------------------------|-------------------------|---------------------|
+| user workspaces      | `1` (reduced iterations)         | `1` (capped grids)      | `1`                 |
+| `*_workspace_test`   | `0` (real searches, `n_like_max`)| unset (full-res)        | unset               |
+
+Per-script `overrides:` still layer **on top of** the selected profile — e.g.
+unset `PYAUTO_SMALL_DATASETS` for full-resolution FITS scripts, keep JAX on for
+`jax_likelihood_functions/` and `jax_substructure/`. The profile sets the floor;
+overrides remain per-script.
+
+The full library env-var surface (canonical entry:
+`PyAutoConf/autoconf/test_mode.py`) is 13 `PYAUTO_*` vars, not the 4 smoke
+defaults:
+
+```
+PYAUTO_TEST_MODE, PYAUTO_SMALL_DATASETS, PYAUTO_FAST_PLOTS, PYAUTO_OUTPUT_MODE,
+PYAUTO_DISABLE_JAX, PYAUTO_SKIP_FIT_OUTPUT, PYAUTO_SKIP_VISUALIZATION,
+PYAUTO_SKIP_CHECKS, PYAUTO_SKIP_LATENTS, PYAUTO_SKIP_WORKSPACE_VERSION_CHECK,
+PYAUTO_LATENT_NAN_INJECT, PYAUTO_DISABLE_IPYTHON_DISPLAY, PYAUTO_LIVE_VIEWER_LOG
+```
+
+plus a few per-script switches outside any yaml default (`PYAUTO_MASS_MODE` /
+`PYAUTO_MASS_FAST`, `JAX_PILOT` / `JAX_PLATFORM_NAME` / `JAX_PLATFORMS`).
+
+**Explicitly NOT Heart's to set.** `config/general.yaml`'s `test:` block
+(`check_likelihood_function`, `lh_timeout_seconds`,
+`disable_positions_lh_inversion_check`) and `version:` toggles are
+workspace-run/user settings. The release validation runs the scripts **as the
+workspace ships them** and does not mutate these. Heart's only version signal is
+the existing `version_skew` check. This `release` profile is an env-var profile,
+not a Heart-owned config mutation.
+
+## Wheel-install requirement (acceptance criteria, wired in M3)
+
+Two verified gaps the M3 integration run MUST close (they are why the report
+carries `profile` and `commit_shas`, so the gate can enforce them):
+
+1. **Test BUILDS, not SOURCE.** Today `workspace-validation.yml` shadows the
+   PyAuto packages with source checkouts via `PYTHONPATH`, so the gating run
+   never touches a wheel — the exact blind spot that let the PyAutoFit `[nss]`
+   git-URL break every TestPyPI upload for weeks. The release run MUST
+   `pip install` the TestPyPI wheels published by the M1 rehearsal and put **no**
+   source on `PYTHONPATH`.
+
+2. **Wheel-based config resolution.** autoconf resolves the *workspace's*
+   `config/` only when scripts run from inside the workspace checkout; a bare
+   wheel falls back to the library's *packaged* defaults. So the run must
+   `pip install` the wheels **but still execute scripts from within the workspace
+   checkout** (for `config/` + `dataset/`), with no source on `PYTHONPATH`.
+
+The integration run also performs `verify_install` A–E against the same wheels.
+
+## How the gate enforces these
+
+`heart/validate.py` records `profile` and per-repo `commit_shas` in
+`validation_report.json`; `heart/readiness.py` then requires, for GREEN:
+
+- `release_ready == true` (no stage failed — else RED),
+- `profile == release` (else YELLOW — a smoke-fidelity run is not a release gate),
+- `commit_shas` matching the current `main` HEADs (else YELLOW — stale source),
+- freshness (a rehearsal older than `VALIDATION_STALE_DAYS` is YELLOW).
+
+Until M3 wires the `release` profile into `workspace-validation.yml`, an ingested
+rehearsal-only report will (correctly) gate YELLOW: the source was built and
+TestPyPI-installed, but not yet exercised at release fidelity.

health_agent/capabilities.yaml

@@ -27,6 +27,7 @@ primary_query:
   persisted_state:
     snapshot: "~/.pyauto-heart/state.json"
     verdict: "~/.pyauto-heart/release_ready.json"
+    validation_report: "~/.pyauto-heart/validation_report.json"
 
 # The authoritative gate. The Health Agent REPORTS this verdict; it must not
 # re-derive it from raw checks. Heart computes it (heart/readiness.py:compute).
@@ -37,6 +38,12 @@ gate:
     yellow: "Mostly healthy. Work may proceed; human review recommended."
     red:    "Blocked. Do not release until the blocking reasons clear."
   rule: "red dominates yellow dominates green; an unknown is YELLOW, never silently green."
+  release_gate: >-
+    GREEN-for-release additionally REQUIRES a fresh, passing validation_report
+    (see the `validate` capability) whose commit_shas match the current `main`
+    HEADs and whose profile == release. Absent/stale/SHA-mismatch/wrong-profile
+    → YELLOW; a failed validation stage → RED. This is the M2 hard gate: no
+    release rehearsal for the current source can no longer be GREEN.
 
 # Continuous checks — cheap, run every <30s tick (heart/tick.sh).
 continuous_checks:
@@ -96,6 +103,31 @@ deep_checks:
     measures: "live HTTP HEAD/GET audit of URLs (Colab/Binder reachability)"
     gate_role: "monitoring only (weekly cloud workflow)"
 
+# Release validation — the third Heart tier (M2). Not a cheap tick, not the
+# existing deep checks: an INGEST-AND-JUDGE capability. Heart defines the report
+# spec and folds already-collected artifacts into it; it NEVER dispatches a build
+# or talks to GitHub (that is the Brain Release Agent's job — see boundary).
+release_validation:
+  - id: validate
+    impl: heart/validate.py
+    cli: "pyauto-heart validate --ingest <artifacts...>"
+    signal: validation_report            # ~/.pyauto-heart/validation_report.json
+    consumes: >-
+      the M1 TestPyPI rehearsal artifact (rehearsal.json / testpypi_version.txt),
+      a {repo: sha} commit_shas.json, and (from M3) the wheel-based integration
+      report.json — collected and handed over by the Brain Release Agent.
+    produces: >-
+      validation_report.json (schema_version 1): release_ready, testpypi_version,
+      profile, per-repo commit_shas, per-stage status, per-project counts,
+      failures, run URLs, timestamp — persisted in Heart state + a
+      validation_history/ archive.
+    gate_role: >-
+      HARD readiness gate. release_ready==false → RED; absent / stale-by-age /
+      commit_shas not matching current main HEADs / profile != release → YELLOW;
+      fresh pass matching current source under the release profile → GREEN.
+    orchestrated_by: "PyAutoBrain Release Agent (agents/release/) — dispatch/poll/download"
+    boundary: "ingest-and-judge only; never dispatches release.yml or workspace-validation.yml"
+
 # GitHub workflows Heart owns (the cloud half of the hybrid CI layer).
 workflows:
   - file: .github/workflows/lib-tests.yml
@@ -123,7 +155,15 @@ operations:
 
 # Boundary the agent must respect (see pyautobuild_boundary_audit.md).
 boundary:
-  heart: "owns all health/readiness checks and the authoritative readiness verdict"
-  brain: "reasons over Heart's outputs (this Health Agent); owns no checks"
+  heart: >-
+    owns all health/readiness checks, the release-validation report spec + ingest,
+    and the authoritative readiness verdict. Ingest-and-judge only: it NEVER
+    dispatches release.yml / workspace-validation.yml or mutates a repo.
+  brain_health: "reasons over Heart's outputs (the Health Agent, read-only); owns no checks; never dispatches"
+  brain_release: >-
+    the Release Agent ORCHESTRATES release validation: dispatches the M1 TestPyPI
+    rehearsal + (M3) the wheel integration run, polls, downloads the artifacts,
+    hands them to `pyauto-heart validate --ingest`, then consults the Health Agent
+    for the verdict. All GitHub dispatch/credentials live here, not in Heart.
   build_hands: "pure executor; runs NO readiness checks; acts only on a GREEN/YELLOW/RED decision"
-  call_chain: "Brain -> Heart (gate) -> Hands (execute)"
+  call_chain: "Mind -> Release Agent (orchestrate) -> Heart (measure) -> Health Agent (judge) -> Hands/Build (promote on GREEN)"

heart/readiness.py

@@ -14,15 +14,20 @@
   ``main``, has uncommitted source changes, or is behind origin; or any workspace
   is pinned AHEAD of its installed library, has a ``general.yaml`` ↔
   ``version.txt`` MISMATCH, or an unparseable (BAD) version; or the deep install
-  verification last reported ``ready == false``.
+  verification last reported ``ready == false``; or the release-validation
+  report last ingested reports ``release_ready == false`` (a stage failed).
 - **YELLOW** (caution) for soft signals: workspace-validation not passing (the
   workspace scripts/notebooks carry standing debt, so this is advisory — never a
   hard block), script-timing regressions, stale open PRs, stale parked scripts, a
-  workspace pinned BEHIND, a stale or never-run install verification, and —
-  crucially — any *unknown* (missing test-run report, a library absent from the
-  snapshot). An unknown is never silently treated as green and never escalated to
-  red.
-- **GREEN** otherwise.
+  workspace pinned BEHIND, a stale or never-run install verification, **no fresh
+  release-validation report for the current source** (absent, stale by age, or
+  whose ``commit_shas`` no longer match the current ``main`` HEADs, or which ran
+  under a profile other than ``release``), and — crucially — any *unknown*
+  (missing test-run report, a library absent from the snapshot). An unknown is
+  never silently treated as green and never escalated to red.
+- **GREEN** otherwise — which now REQUIRES a fresh, passing release-validation
+  report matching the current source under the ``release`` profile, not just an
+  absence of red signals.
 
 Red dominates yellow structurally: reasons are collected into separate lists
 and ``verdict = red if red_reasons else yellow if yellow_reasons else green``.
@@ -88,6 +93,12 @@
     "install_unknown": (10, 10),
     "test_stale": (10, 10),
     "ws_ci": (20, 60),
+    "validation_failed": (40, 40),
+    "validation_absent": (15, 15),
+    "validation_stale_sha": (15, 15),
+    "validation_unknown": (12, 12),
+    "validation_profile": (12, 12),
+    "validation_stale": (10, 10),
 }
 
 
@@ -112,6 +123,29 @@ def _as_int(v: Any, default: int = 0) -> int:
 INSTALL_STALE_DAYS = 14
 # workspace-validation test run older than this many days is treated as stale.
 TEST_STALE_DAYS = 10
+# a release-validation rehearsal older than this many days is stale (YELLOW),
+# even if its commit_shas still match — releases move fast enough that a
+# week-old rehearsal warrants a fresh run.
+VALIDATION_STALE_DAYS = 7
+
+# The library repos whose current `main` HEAD must match a validation report's
+# recorded commit_shas for the release-validation gate to go GREEN.
+_GATE_SHA_LIBS = DEFAULT_LIBRARIES
+
+
+def _sha_eq(a: Any, b: Any) -> bool:
+    """True if two commit SHAs refer to the same commit (prefix-tolerant).
+
+    ci_status records a short (7-char) HEAD sha while a report may carry the
+    full 40-char sha (or vice versa), so compare on the shorter length. A match
+    needs at least 7 significant hex chars — an empty/too-short value never
+    matches (it is treated as *unknown*, never a silent pass)."""
+    sa = str(a or "").strip().lower()
+    sb = str(b or "").strip().lower()
+    n = min(len(sa), len(sb))
+    if n < 7:
+        return False
+    return sa[:n] == sb[:n]
 
 
 def _parse_ts(ts: Any) -> datetime.datetime | None:
@@ -287,6 +321,72 @@ def hit(key: str, n: int = 1) -> None:
         yellow.append("install verification not run")
         hit("install_unknown")
 
+    # --- release-validation gate (hard) -------------------------------------
+    # GREEN-for-release now REQUIRES a fresh, passing validation_report whose
+    # commit_shas match the current `main` HEADs and whose profile == release.
+    # This is the M2 gate: the report proves the exact source about to ship was
+    # built, published to TestPyPI, installed from the wheel, and exercised at
+    # release fidelity. Absent/stale/source-not-matching → YELLOW ("no release
+    # rehearsal for current source"); failing → RED. Pass/fail (release_ready)
+    # is the RED axis; fidelity+freshness (profile / commit_shas / age) is the
+    # YELLOW axis — a passing-but-stale report is a caution, not a blocker.
+    vr = snapshot.get("validation_report")
+    if isinstance(vr, dict) and vr:
+        ready = vr.get("release_ready")
+        if ready is False:
+            failed_stages = [
+                n for n, s in (vr.get("stages") or {}).items()
+                if isinstance(s, dict) and s.get("status") == "fail"
+            ]
+            red.append(
+                "release validation FAILED"
+                + (f" (stage {', '.join(failed_stages)})" if failed_stages else "")
+            )
+            hit("validation_failed")
+        elif ready is True:
+            commit_shas = vr.get("commit_shas") or {}
+            mismatched: list[str] = []
+            confirmed = 0
+            for lib in _GATE_SHA_LIBS:
+                want = commit_shas.get(lib)
+                cur = ((repos.get(lib) or {}).get("ci_status") or {}).get("head_sha")
+                if not want or not cur:
+                    continue  # can't confirm this repo (unknown), don't over-penalise
+                if _sha_eq(want, cur):
+                    confirmed += 1
+                else:
+                    mismatched.append(lib)
+            profile = str(vr.get("profile") or "").strip().lower()
+            if not commit_shas:
+                yellow.append("release validation source unconfirmed (no commit_shas)")
+                hit("validation_unknown")
+            elif mismatched:
+                yellow.append(
+                    "release validation stale: source moved since rehearsal ("
+                    + ", ".join(mismatched) + ")"
+                )
+                hit("validation_stale_sha")
+            elif confirmed == 0:
+                yellow.append("release validation source unconfirmed (current HEADs unknown)")
+                hit("validation_unknown")
+            elif profile != "release":
+                yellow.append(
+                    f"release validation profile '{vr.get('profile') or '?'}' is not 'release'"
+                )
+                hit("validation_profile")
+            else:
+                age = _age_days(vr.get("ts"), ref)
+                if age is not None and age > VALIDATION_STALE_DAYS:
+                    yellow.append(f"release validation stale ({int(age)}d old)")
+                    hit("validation_stale")
+                # else: fresh, passing, matching, release profile → GREEN-eligible
+        else:
+            yellow.append("release validation status unknown")
+            hit("validation_unknown")
+    else:
+        yellow.append("no release validation for current source")
+        hit("validation_absent")
+
     # --- script timing (YELLOW) ---
     timing = snapshot.get("script_timing", {}) or {}
     if _as_int(timing.get("red_count", 0)) > 0:

heart/state.py

@@ -84,6 +84,7 @@ def aggregate() -> dict[str, Any]:
         "version_skew": _read_json_or_default(HEART_STATE_DIR / "version_skew.json", {}),
         "verify_install": _read_json_or_default(HEART_STATE_DIR / "verify_install.json", {}),
         "url_check": _read_json_or_default(HEART_STATE_DIR / "url_check.json", {}),
+        "validation_report": _read_json_or_default(HEART_STATE_DIR / "validation_report.json", {}),
     }
     atomic_write_json(HEART_STATE_FILE, snapshot)
     return snapshot