Skip to content

docs(337): make the verification claim accurate (relabel + narrow) - #350

Merged
avrabe merged 1 commit into
mainfrom
docs/337-verification-claim-accuracy
Jul 15, 2026
Merged

docs(337): make the verification claim accurate (relabel + narrow)#350
avrabe merged 1 commit into
mainfrom
docs/337-verification-claim-accuracy

Conversation

@avrabe

@avrabe avrabe commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Corrects the README verification claim to match the actual evidence (the claim-verification "relabel-and-gate" pattern #337 asks for). Grounded in proofs/STATUS.md and meld-core/src/abi_proofs.rs.

Why the old claim overstated

  • Flat always-green Formally_Verified badge — unlinked, reflects no job.
  • "Every transformation carries mechanized proofs covering parsing, resolution, merging, and adapter correctness" — but the coverage matrix shows parser / rewriter / segments / attestation are placeholders, orchestration is unproven, semantic preservation is a forward simulation only, and one FACT adapter lemma is Admitted.
  • Kani harnesses are VERIFICATION SUCCESSFUL locally (SR-40), not gated in CI (shared runners lack the Kani toolchain); ABI size/align + aggregate-padding contracts fell back to unit tests.

What changed (all README)

  • Badge → linked Formal_Methods: Rocq + Kani (partial)#formal-verification (informational, not a false CI-gate).
  • Tagline → narrowed to the core stages actually proven, with a scope caveat + link.
  • Formal Verification section → rewritten to state the 350 closed Rocq proofs honestly: real/substantial on resolver·merger·adapter·spec, but not whole-pipeline; forward-simulation only; the one Admitted lemma called out; Kani local-not-CI; unit-test fallback noted. Links proofs/STATUS.md.

This neither over- nor under-sells — it credits the genuine 350-proof body of work while scoping it accurately.

Not addressed here

#337(b) — gating the converging Kani harnesses in CI — needs the Kani toolchain on the shared smithy runners (an infra change), so it's a follow-up, not this docs PR.

Refs #337.

🤖 Generated with Claude Code

The flat always-green `Formally_Verified` badge and the "every transformation
carries mechanized proofs covering parsing, resolution, merging, and adapter
correctness" tagline overstated the actual evidence (grounded in
proofs/STATUS.md + abi_proofs.rs):
- Kani harnesses (abi/resolver/merger) are VERIFICATION SUCCESSFUL LOCALLY
  (SR-40) but NOT gated in CI (shared runners lack the Kani toolchain); some
  ABI size/align + aggregate-padding contracts fell back to unit tests.
- Rocq: 350 closed proofs on resolver/merger/adapter/spec (real, substantial),
  but parser/rewriter/segments/attestation are placeholders, orchestration is
  unproven, semantic preservation is a FORWARD simulation only, and one FACT
  adapter lemma is Admitted — so "every transformation … behaves identically"
  overstates.

Fix (claim-verification relabel):
- Badge → linked `Formal_Methods: Rocq + Kani (partial)` pointing at the
  Formal Verification section (informational, not a false CI-gate claim).
- Tagline → narrowed to the core stages actually proven + a scope caveat.
- Formal Verification section → rewritten to state the 350-proof coverage
  honestly (what's proved vs placeholder, forward-only simulation, the one
  Admitted lemma, Kani local-not-CI), linking proofs/STATUS.md.

Not addressed here (infra): #337(b) gating the converging Kani harnesses in CI
needs the Kani toolchain on the shared runners — a smithy-infra change.

Refs #337.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

LS-N verification gate

59/59 approved LS entries verified

count
Passed (≥1 test, all green) 59
Failed (≥1 test failure) 0
Missing (no ls_*_NN_* test found) 0

Approved loss-scenarios.yaml entries are expected to have a
regression test named ls_<letter>_<num>_* (e.g. LS-A-11
ls_a_11_*). The gate runs each prefix via cargo test --lib --no-fail-fast and aggregates pass/fail/missing.

Failed LS entries

(none)

Missing regression tests

(none)

Updated automatically by tools/post_verification_comment.py.
Source of truth: safety/stpa/loss-scenarios.yaml.

@avrabe
avrabe merged commit 3218503 into main Jul 15, 2026
8 checks passed
@avrabe
avrabe deleted the docs/337-verification-claim-accuracy branch July 15, 2026 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant