You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Owner ruled the six open D1 questions (#69) as **a1, b1, c1, d1, e2,
f1**. This PR lands the whole ruling in one change, per the issue's
done-condition ("rulings recorded here, protocol amended to v2.1 with
examples updated").
## What's in it
- **`vexometer/docs/EFFICACY-PROTOCOL.adoc` → v2.1** — every ruled
semantic written into the normative text, all JSON examples regenerated
from the binary (doc == tool is enforced by exact-value equality in the
test suite):
- **a1** — a target metric with `B_m = 0` gets `G_m := 0`, so it cannot
improve and the verdict is `reject_null`; the report lists offenders in
a diagnosability warning. Zero-baseline *collateral* metrics stay fully
protected.
- **b1** — per-probe identity gate is normative when `probes.results`
exists in both measurements (at most one baseline-passing probe may fail
after; newly-passing probes buy nothing back); aggregate pass-rate is
the degraded fallback. `capability.probes_regressed` records which gate
applied.
- **c1** — all-targets rule: every declared target must improve or the
verdict is `reject_null`.
- **d1** — plural `frontier_records`, one per-metric record per target
in target order; length mismatch is a hard error; the singular
`frontier_record` key fails validation.
- **e2** — new `lift` subcommand: mechanical v1→v2.1 lift, v1 fields
verbatim, missing evidence as explicit `null` (never synthesised),
`lifted_from` marker, `verdict: "unverified"` reserved for lifted
reports. Normative spec + worked example in the doc.
- **f1** — held-out scenario-set registry
(`vexometer/data/scenario_sets/registry.json`, ships empty — no corpus
exists yet, no invented hashes); `validate --scenario-registry` rejects
scoring on a tuning partition and unknown scenario sets, covering both
efficacy reports and frontier attempts. Registering the first partition
is a precondition for the first satellite evaluation (D6).
- **`vexometer-efficacy`** — every `AwaitingRuling` refusal replaced
with the ruled semantics; exit code 2 (the "open D1 question" refusal)
retired, codes now 0/1/3; the renamed `--frontier-record` flag gets an
explicit error pointing at `--frontier-records`.
- **Justfile** — `efficacy-lift` recipe; the protocol's stale "these
recipes do not yet exist" sentence replaced with the real tooling list.
- **README/ROADMAP** updated; **trust manifests regenerated in-PR** per
the manifest contract.
## Verification
- `cargo test`: 29/29 (protocol examples are the fixtures; report + lift
roundtrips assert exact JSON equality with the doc)
- `cargo clippy --all-targets -- -D warnings` clean, `cargo fmt --check`
clean
- `just must-all` green, `just trust-manifest-verify` green, `just
--evaluate` parses
Closes#69
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
0 commit comments