diff --git a/README.md b/README.md index f615e75..b8d2012 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Open Institutional Compiler -- **Status:** BOUNDED_REFERENCE_IMPLEMENTATION — scoped independent Gate G validation passed for candidate `a2b5053771ce510fb35ce09f3e99f545c21ac20e` and merged to `main` as `c4a325c551ce8904dfcc5b9fe81b05109726a334` +- **Status:** BOUNDED_REFERENCE_IMPLEMENTATION — scoped independent Gate G validation passed for candidate `a2b5053771ce510fb35ce09f3e99f545c21ac20e` and merged to `main` as `c4a325c551ce8904dfcc5b9fe81b05109726a334`; a later claim-control continuity repair, candidate `4576fea067d33c2c0f8e8fc2d49bb506c52b2f1c`, was merged as `8ebac66965997748061d8cc0f1bfde73cb7b216a`, which independent review does not cover - **Bootstrap date:** 2026-07-29 - **Governing design:** TDD-OIC-001 v1.1 @@ -57,6 +57,17 @@ byte-identical offline demo digest. Pull request 40 merged that exact tree into merged at 2026-09-06T08:14:03Z. That result carries the same twelve exclusions and the same ceilings, is scoped to that exact candidate and tree, and is not validation of any later commit. +Pull request 41 subsequently merged candidate +`4576fea067d33c2c0f8e8fc2d49bb506c52b2f1c` (tree +`6e450750d7b6b1d1f0b493b050ed7d866d42b264`) into `main` as merge commit +`8ebac66965997748061d8cc0f1bfde73cb7b216a`, first parent +`c4a325c551ce8904dfcc5b9fe81b05109726a334`. That candidate changes claim-control +text only: it adds the subject term `implementation` to the README unscoped-claim +alternation and adds the matching regression case, closing SAK001C-F01/SAK001-F01 +pending independent confirmation. It changes no runtime, verifier, matrix, +benchmark or claim content. Independent review does not cover merge commit +`8ebac66965997748061d8cc0f1bfde73cb7b216a` or candidate +`4576fea067d33c2c0f8e8fc2d49bb506c52b2f1c`. See [`CAPABILITY_MATRIX.json`](docs/capabilities/CAPABILITY_MATRIX.json) for exact provenance and ceilings. NVIDIA is NOT_QUALIFIED; Canada redistribution is UNRESOLVED; Ontology 007R1 is unexecuted and execution-unauthorized. No model accuracy or legal validity is claimed. @@ -354,6 +365,51 @@ These results verify repository infrastructure. They are not semantic OIC-Bench results. The benchmark preflight metrics remain proposed or preregistered and not measured. Experimental branch results are not accepted benchmark evidence. +### What has actually been measured + +Nothing in OIC-Bench has been measured. Every figure below is repository or +bounded-implementation evidence, scoped to the exact candidate that produced it. + +- Independent Gate F repository validation, candidate + `c0108a7a80585d6f5732407d4904ba815073ecd2`, tree + `1d12b17aad7977c939090909171183be166cfd50`: 1714 passed, 0 failed, 0 errors, + 1 declared skip, 93.5% coverage. +- Independent Gate G validation, candidate + `a2b5053771ce510fb35ce09f3e99f545c21ac20e`, tree + `b8e31ec4786a2fd1aca976a6ff047deeee63ef15`: 1720 passed, 0 failed, 0 errors, + 1 declared skip, 93.5% coverage, zero critical and zero major findings. +- Deterministic offline path, candidate + `c0108a7a80585d6f5732407d4904ba815073ecd2`, tree + `1d12b17aad7977c939090909171183be166cfd50`, and again at candidate + `a2b5053771ce510fb35ce09f3e99f545c21ac20e`, tree + `b8e31ec4786a2fd1aca976a6ff047deeee63ef15`: two separate-process runs emit + byte-identical canonical JSON, SHA-256 + `0f9d01bb0dfc488505e027ac7bd8aecf869578e379b5a977cd9d642f2101a39a`. At the + second of those candidates both runs executed with no network interfaces and no + DNS available, so the offline property is demonstrated rather than merely + supported. +- Bounded synthetic path composition, candidate + `a2b5053771ce510fb35ce09f3e99f545c21ac20e`, tree + `b8e31ec4786a2fd1aca976a6ff047deeee63ef15`: one synthetic fictional sentence + yields two divergent candidate units, eleven provisional interpretation slots, + one preserved unresolved reference, one supplied synthetic authority record, and + three refusal paths. Institutional admission is false. These are counts from a + single fixture. They are not a source-support rate, an ambiguity measurement, or + any other benchmark quantity. +- Infrastructure verification, candidate + `a2b5053771ce510fb35ce09f3e99f545c21ac20e`, tree + `b8e31ec4786a2fd1aca976a6ff047deeee63ef15`: schemas 9/9, bootstrap 52/52, + falsification harness 4/4, and manifest verification deliberately `INCOMPLETE` + at exit 3. + +Test counts measure the test suite. They do not measure whether compiled controls +behave correctly, and they are not benchmark results. Within the authorized OIC evidence universe examined by +OIC-NIM-EVIDENCE-CROSSWALK-001 as of 2026-09-07, no practitioner study, baseline +arm, human-time instrumentation or adjudicated behavioural-quality scale is +present, so no comparative statement is supported from that evidence. That is a +statement about that bounded evidence set on that date, not a claim about any +work outside it. + ## Development roadmap The roadmap is evidence-gated, not schedule-driven. No stage is complete diff --git a/STATUS.md b/STATUS.md index 227172b..74cd9a0 100644 --- a/STATUS.md +++ b/STATUS.md @@ -34,6 +34,20 @@ two byte-identical offline demo runs with SHA-256 byte-identical to the validated tree. Each result above is scoped to the exact commit and tree that produced it and to nothing later. +Pull request 41 subsequently merged candidate +`4576fea067d33c2c0f8e8fc2d49bb506c52b2f1c`, tree +`6e450750d7b6b1d1f0b493b050ed7d866d42b264`, into `main` as merge commit +`8ebac66965997748061d8cc0f1bfde73cb7b216a`, first parent +`c4a325c551ce8904dfcc5b9fe81b05109726a334`. That candidate changes claim-control +text only: it adds the subject term `implementation` to the README unscoped-claim +alternation and adds the matching case to the internal regression tuple, closing +SAK001C-F01/SAK001-F01 pending independent confirmation. It changes no runtime, +verifier, matrix, benchmark or claim content, and the merged tree is byte-identical +to the candidate tree. Independent review does not cover merge commit +`8ebac66965997748061d8cc0f1bfde73cb7b216a` or candidate +`4576fea067d33c2c0f8e8fc2d49bb506c52b2f1c`; the Gate F and Gate G results above +remain scoped to the exact earlier candidates that produced them. + NVIDIA: NOT_QUALIFIED and excluded from the demo. Canada redistribution: UNRESOLVED. Ontology 007R1: unexecuted and execution-unauthorized. Institutional-IR closure: UNESTABLISHED. Production compilation and runtime authorization: UNESTABLISHED. Negative-stability @@ -61,5 +75,52 @@ production readiness, public quality, universal novelty, or legal compliance. **Broader production semantic gate:** BLOCKED. Historical NOT OPEN receipts remain unchanged; the active capability matrix supersedes only the bounded synthetic surface. +## Bounded results established, and what they are not + +Nothing in OIC-Bench has been measured. Every figure below is repository or +bounded-implementation evidence, scoped to the exact candidate that produced it. +All eight preregistered OIC-Bench rows in `README.md` remain TARGET - NOT MEASURED, +and the provisional comparative target remains PROVISIONAL TARGET - NOT MEASURED - +NOT CALIBRATED. + +- Independent Gate F repository validation, candidate + `c0108a7a80585d6f5732407d4904ba815073ecd2`, tree + `1d12b17aad7977c939090909171183be166cfd50`: 1714 passed, 0 failed, 0 errors, + 1 declared skip, 93.5% coverage. +- Independent Gate G validation, candidate + `a2b5053771ce510fb35ce09f3e99f545c21ac20e`, tree + `b8e31ec4786a2fd1aca976a6ff047deeee63ef15`: 1720 passed, 0 failed, 0 errors, + 1 declared skip, 93.5% coverage, zero critical and zero major findings. +- Deterministic offline path, candidate + `c0108a7a80585d6f5732407d4904ba815073ecd2`, tree + `1d12b17aad7977c939090909171183be166cfd50`, and again at candidate + `a2b5053771ce510fb35ce09f3e99f545c21ac20e`, tree + `b8e31ec4786a2fd1aca976a6ff047deeee63ef15`: two separate-process runs emit + byte-identical canonical JSON, SHA-256 + `0f9d01bb0dfc488505e027ac7bd8aecf869578e379b5a977cd9d642f2101a39a`. At the + second of those candidates both runs executed with no network interfaces and no + DNS available. +- Bounded synthetic path composition, candidate + `a2b5053771ce510fb35ce09f3e99f545c21ac20e`, tree + `b8e31ec4786a2fd1aca976a6ff047deeee63ef15`: one synthetic fictional sentence + yields two divergent candidate units, eleven provisional interpretation slots, + one preserved unresolved reference, one supplied synthetic authority record, and + three refusal paths, with institutional admission false. These are counts from a + single fixture, not a source-support rate, an ambiguity measurement, or any other + benchmark quantity. +- Infrastructure verification, candidate + `a2b5053771ce510fb35ce09f3e99f545c21ac20e`, tree + `b8e31ec4786a2fd1aca976a6ff047deeee63ef15`: schemas 9/9, bootstrap 52/52, + falsification harness 4/4, and manifest verification deliberately INCOMPLETE at + exit 3. + +Test counts measure the test suite. They do not measure whether compiled controls +behave correctly, and they are not benchmark results. Within the authorized OIC evidence universe examined by +OIC-NIM-EVIDENCE-CROSSWALK-001 as of 2026-09-07, no practitioner study, baseline +arm, human-time instrumentation or adjudicated behavioural-quality scale is +present, so no comparative statement is supported from that evidence. That is a +statement about that bounded evidence set on that date, not a claim about any +work outside it. + The reference path does not confer real institutional authority or runtime permission. After this work order, another deposited authorization plus explicit execution signal is required. diff --git a/tests/contract/test_claims_discipline.py b/tests/contract/test_claims_discipline.py index b70ff2b..ed23bb3 100644 --- a/tests/contract/test_claims_discipline.py +++ b/tests/contract/test_claims_discipline.py @@ -567,3 +567,355 @@ def test_matrix_gate_f_exclusion_mutations_fail_closed(repo_root: Path, mutation exclusions.append("production readiness") with pytest.raises(module.GateEvidenceError, match="evidence forged"): module.validate_independent_validation_evidence(evidence) + + +#: Anchors for the PR #41 continuity merge. These are deliberately literal rather than read +#: from the capability matrix: the matrix records the Gate F and Gate G candidates and is +#: outside the authorized path set for the work order that recorded this merge, so binding +#: the front doors to it here would couple two independently governed artifacts. +POST_MERGE_CONTINUITY_ANCHORS = ( + "8ebac66965997748061d8cc0f1bfde73cb7b216a", + "4576fea067d33c2c0f8e8fc2d49bb506c52b2f1c", + "c4a325c551ce8904dfcc5b9fe81b05109726a334", + "6e450750d7b6b1d1f0b493b050ed7d866d42b264", +) + +#: Prose the continuity merge record must carry in both front doors. +POST_MERGE_CONTINUITY_PHRASES = ( + "pull request 41", + "sak001c-f01/sak001-f01 pending independent confirmation", + "independent review does not cover merge commit", +) + +#: Misclassified review wording. The claim controls read "independently validated" as an +#: unscoped assertion, so the front doors must phrase non-coverage a different way. +POST_MERGE_CONTINUITY_MISDESCRIPTIONS = ( + "is not independently validated", + "are not independently validated", +) + +#: References that identify the continuity candidate published as PR 41. +CONTINUITY_CANDIDATE_REFERENCES = ( + "4576fea067d33c2c0f8e8fc2d49bb506c52b2f1c", + "pull request 41", + "pr 41", +) + +#: Verbs that assert a finding was disposed of by the text they appear in. +CLOSURE_MARKERS = ("clos", "resolv", "fixes", "fixed") + +#: Characters of context inspected either side of a GG001-M01 mention. Wide enough to span a +#: sentence and its neighbour, so a claim split across a clause boundary is still caught. +GG001_CONTEXT = 240 + +#: The eight preregistered OIC-Bench benchmarks. Every one must remain an unmeasured target. +OIC_BENCH_ROWS = ( + "Source-supported executable fields", + "Unsupported executable-field rate", + "Unknown-to-false conversions", + "Authority Reconstruction F1", + "Ambiguity recall", + "False-resolution rate", + "Behavioral conformance", + "Change-impact recall", +) + +#: Markers for the bounded-results disclosure. Each states a fact that is true of the +#: bounded evidence and false of an OIC-Bench measurement. +BOUNDED_DISCLOSURE_PHRASES = ( + "nothing in oic-bench has been measured", + "test counts measure the test suite", + "they are not benchmark results", +) + +#: Headings that open the bounded-results disclosure in each front door. +BOUNDED_DISCLOSURE_HEADINGS = { + "README.md": "### What has actually been measured", + "STATUS.md": "## Bounded results established, and what they are not", +} + +#: Producing candidate commit to its tree. Every numeric bounded-result bullet must name a +#: candidate from this mapping together with that candidate's tree, so no figure can travel +#: without the exact repository state that produced it. +BOUNDED_RESULT_BINDINGS = { + "c0108a7a80585d6f5732407d4904ba815073ecd2": "1d12b17aad7977c939090909171183be166cfd50", + "a2b5053771ce510fb35ce09f3e99f545c21ac20e": "b8e31ec4786a2fd1aca976a6ff047deeee63ef15", +} + +#: The absence statement must be bounded to the evidence universe and date that support it. +PRACTITIONER_BOUNDING_PHRASES = ( + "within the authorized oic evidence universe examined by oic-nim-evidence-crosswalk-001 " + "as of 2026-09-07", + "not a claim about any work outside it", +) + +#: Substrings of the bounding phrases that survive on a single source line. The phrases above +#: are asserted against whitespace-normalized text so hard wrapping cannot defeat them; these +#: anchors exist so a mutation can delete a bound from the raw file. +PRACTITIONER_BOUNDING_LINE_ANCHORS = ( + "OIC-NIM-EVIDENCE-CROSSWALK-001 as of 2026-09-07", + "not a claim about any", +) + +#: Unbounded absolute phrasings of the same statement. These assert something about all work +#: everywhere, which no evidence in the authorized universe supports. +PRACTITIONER_UNBOUNDED_WORDINGS = ( + "exists, so no comparative statement is supported in either direction", + "exists, so no comparative statement of any kind is supported", +) + + +def _bounded_result_bullets(text: str, relpath: str) -> list[str]: + """Return the bullet items of the bounded-results disclosure, each joined to one line.""" + heading = BOUNDED_DISCLOSURE_HEADINGS[relpath] + assert heading in text, f"missing bounded-results heading in {relpath}" + section = text.split(heading, 1)[1] + for line in section.splitlines(): + if line.startswith("#"): + section = section.split("\n" + line, 1)[0] + break + bullets: list[str] = [] + for line in section.splitlines(): + if line.startswith("- "): + bullets.append(line[2:].strip()) + elif line.startswith(" ") and bullets: + bullets[-1] += " " + line.strip() + assert bullets, f"no bounded-results bullets found in {relpath}" + return bullets + + +def _assert_bounded_results_are_candidate_bound(text: str, relpath: str) -> None: + """Every bounded-result bullet must carry its producing candidate commit and tree.""" + for bullet in _bounded_result_bullets(text, relpath): + normalized = " ".join(bullet.lower().replace("`", "").split()) + cited = [c for c in BOUNDED_RESULT_BINDINGS if c in normalized] + assert cited, f"bounded-result bullet names no producing candidate: {bullet[:80]!r}" + for candidate in cited: + tree = BOUNDED_RESULT_BINDINGS[candidate] + assert tree in normalized, ( + f"bounded-result bullet cites candidate {candidate} without its tree {tree}" + ) + + +def _assert_practitioner_absence_is_bounded(text: str) -> None: + """The practitioner/baseline absence statement must name its evidence universe and date.""" + normalized = " ".join(text.lower().replace("*", "").split()) + for phrase in PRACTITIONER_BOUNDING_PHRASES: + assert phrase in normalized, f"practitioner absence statement is unbounded: {phrase}" + for phrase in PRACTITIONER_UNBOUNDED_WORDINGS: + assert phrase not in normalized, ( + f"practitioner absence statement asserts an unbounded universal: {phrase}" + ) + + +def _assert_post_merge_continuity(text: str) -> None: + """Require the PR #41 continuity merge to be recorded, scoped and not overstated.""" + normalized = " ".join(text.lower().replace("*", "").split()) + for anchor in POST_MERGE_CONTINUITY_ANCHORS: + assert anchor in normalized, f"missing continuity anchor: {anchor}" + for phrase in POST_MERGE_CONTINUITY_PHRASES: + assert phrase in normalized, f"missing continuity statement: {phrase}" + for phrase in POST_MERGE_CONTINUITY_MISDESCRIPTIONS: + assert phrase not in normalized, f"continuity merge is misdescribed: {phrase}" + _assert_gg001_is_not_attributed_to_the_continuity_candidate(normalized) + + +def _assert_gg001_is_not_attributed_to_the_continuity_candidate(normalized: str) -> None: + """Reject only a contextual misattribution of GG001-M01 to the continuity candidate. + + GG001-M01 is a real Gate G finding and the repository is free to discuss it accurately. + What must fail closed is text that credits candidate 4576fea/PR 41 with disposing of it, + because that candidate closes the state-acknowledgment finding SAK001C-F01/SAK001-F01. + A bare prohibition on the identifier would forbid honest history, so the control is + scoped to co-occurrence of the finding, a continuity reference and a closure verb. + """ + start = 0 + while True: + found = normalized.find("gg001-m01", start) + if found < 0: + return + start = found + len("gg001-m01") + window = normalized[max(0, found - GG001_CONTEXT) : start + GG001_CONTEXT] + if not any(marker in window for marker in CLOSURE_MARKERS): + continue + cited = [ref for ref in CONTINUITY_CANDIDATE_REFERENCES if ref in window] + assert not cited, ( + "continuity merge is misdescribed: GG001-M01 is attributed to the continuity " + f"candidate near {cited[0]!r}" + ) + + +def _assert_bounded_results_disclosure(text: str) -> None: + """Require the bounded-results disclosure and refuse a benchmark reading of it.""" + normalized = " ".join(text.lower().replace("*", "").split()) + for phrase in BOUNDED_DISCLOSURE_PHRASES: + assert phrase in normalized, f"missing bounded-results disclosure: {phrase}" + for candidate, passed in ( + ("c0108a7a80585d6f5732407d4904ba815073ecd2", "1714 passed"), + ("a2b5053771ce510fb35ce09f3e99f545c21ac20e", "1720 passed"), + ): + assert candidate in normalized, f"bounded result is unscoped: missing {candidate}" + assert passed in normalized, f"bounded result is missing its count: {passed}" + + +@pytest.mark.parametrize("relpath", ("README.md", "STATUS.md")) +def test_post_merge_continuity_is_recorded_in_both_front_doors( + repo_root: Path, relpath: str +) -> None: + """Both front doors must record the PR #41 merge without overstating its review status. + + Fail-closed coverage: removing any anchor, dropping the non-coverage statement, or + reattributing the candidate to Gate G finding GG001-M01 must each fail. + """ + text = (repo_root / relpath).read_text(encoding="utf-8") + _assert_post_merge_continuity(text) + + for anchor in POST_MERGE_CONTINUITY_ANCHORS: + stripped = text.replace(anchor, "0" * 40) + with pytest.raises(AssertionError, match="missing continuity anchor"): + _assert_post_merge_continuity(stripped) + + dropped = re.sub( + r"Independent review does not cover merge commit", + "Review notes", + text, + flags=re.IGNORECASE, + ) + assert dropped != text + with pytest.raises(AssertionError, match="missing continuity statement"): + _assert_post_merge_continuity(dropped) + + misattributed = text + "\nGG001-M01 is closed by pull request 41.\n" + with pytest.raises(AssertionError, match="continuity merge is misdescribed"): + _assert_post_merge_continuity(misattributed) + + misclassified = text + "\nThe current main is not independently validated.\n" + with pytest.raises(AssertionError, match="continuity merge is misdescribed"): + _assert_post_merge_continuity(misclassified) + + +@pytest.mark.parametrize("relpath", ("README.md", "STATUS.md")) +def test_bounded_results_disclosure_is_present_and_scoped(repo_root: Path, relpath: str) -> None: + """Bounded results must be disclosed as bounded, with each figure bound to its candidate.""" + text = (repo_root / relpath).read_text(encoding="utf-8") + _assert_bounded_results_disclosure(text) + + for phrase in BOUNDED_DISCLOSURE_PHRASES: + removed = re.sub(re.escape(phrase), "removed", text, flags=re.IGNORECASE) + assert removed != text, f"disclosure phrase not found for mutation: {phrase}" + with pytest.raises(AssertionError, match="missing bounded-results disclosure"): + _assert_bounded_results_disclosure(removed) + + unscoped = text.replace("c0108a7a80585d6f5732407d4904ba815073ecd2", "an earlier candidate") + with pytest.raises(AssertionError, match="bounded result is unscoped"): + _assert_bounded_results_disclosure(unscoped) + + +def test_oic_bench_rows_remain_unmeasured_targets(repo_root: Path) -> None: + """All eight preregistered rows stay TARGET - NOT MEASURED, and the comparative target too. + + Promotion of any row is the single most consequential unsupported claim this repository + could publish, so it fails closed here rather than depending on review. + """ + readme = (repo_root / "README.md").read_text(encoding="utf-8") + + for row in OIC_BENCH_ROWS: + pattern = re.compile( + rf"^\|\s*{re.escape(row)}\s*\|[^|]*\|\s*TARGET - NOT MEASURED\s*\|$", + re.MULTILINE, + ) + assert pattern.search(readme), f"OIC-Bench row is not an unmeasured target: {row}" + + assert "PROVISIONAL TARGET - NOT MEASURED - NOT CALIBRATED" in readme, ( + "the provisional comparative target lost its uncalibrated status" + ) + + promoted = readme.replace( + "| Ambiguity recall | >=0.85 | TARGET - NOT MEASURED |", + "| Ambiguity recall | >=0.85 | MEASURED - 0.91 |", + ) + assert promoted != readme + pattern = re.compile( + r"^\|\s*Ambiguity recall\s*\|[^|]*\|\s*TARGET - NOT MEASURED\s*\|$", re.MULTILINE + ) + assert pattern.search(promoted) is None, "row promotion was not detected" + + +@pytest.mark.parametrize("relpath", ("README.md", "STATUS.md")) +def test_gg001_control_permits_history_and_rejects_misattribution( + repo_root: Path, relpath: str +) -> None: + """GG001-M01 may be discussed accurately; crediting the continuity candidate must fail. + + The predecessor control banned the bare identifier, which would have forbidden honest + history. This proves both halves: an accurate historical mention passes, and every way of + attributing the finding to candidate 4576fea/PR 41 fails closed. + """ + text = (repo_root / relpath).read_text(encoding="utf-8") + _assert_post_merge_continuity(text) + + historical = text + ( + "\nGate G recorded finding GG001-M01 against candidate " + "a2b5053771ce510fb35ce09f3e99f545c21ac20e as MINOR and non-blocking.\n" + ) + _assert_post_merge_continuity(historical) + + unrelated_closure = text + ( + "\nGG001-M01 was resolved by a later work order that this document does not describe.\n" + ) + _assert_post_merge_continuity(unrelated_closure) + + for reference in CONTINUITY_CANDIDATE_REFERENCES: + misattributed = text + f"\nGG001-M01 is closed by {reference}.\n" + with pytest.raises(AssertionError, match="GG001-M01 is attributed"): + _assert_post_merge_continuity(misattributed) + + reversed_order = text + f"\n{reference} closes finding GG001-M01.\n" + with pytest.raises(AssertionError, match="GG001-M01 is attributed"): + _assert_post_merge_continuity(reversed_order) + + +@pytest.mark.parametrize("relpath", ("README.md", "STATUS.md")) +def test_every_bounded_result_bullet_is_candidate_and_tree_bound( + repo_root: Path, relpath: str +) -> None: + """No bounded figure may travel without the exact candidate commit and tree behind it.""" + text = (repo_root / relpath).read_text(encoding="utf-8") + _assert_bounded_results_are_candidate_bound(text, relpath) + + bullets = _bounded_result_bullets(text, relpath) + assert len(bullets) >= 5, f"expected the full bounded-results list in {relpath}" + for candidate, tree in BOUNDED_RESULT_BINDINGS.items(): + assert any(candidate in b for b in bullets), f"no bullet cites candidate {candidate}" + assert any(tree in b for b in bullets), f"no bullet cites tree {tree}" + + for tree in BOUNDED_RESULT_BINDINGS.values(): + stripped = text.replace(f", tree\n `{tree}`", "").replace(f"tree\n `{tree}`", "") + assert stripped != text, f"tree citation not found for mutation: {tree}" + with pytest.raises(AssertionError, match="without its tree"): + _assert_bounded_results_are_candidate_bound(stripped, relpath) + + for candidate in BOUNDED_RESULT_BINDINGS: + anonymised = text.replace(candidate, "an earlier candidate") + with pytest.raises(AssertionError, match="names no producing candidate|without its tree"): + _assert_bounded_results_are_candidate_bound(anonymised, relpath) + + +@pytest.mark.parametrize("relpath", ("README.md", "STATUS.md")) +def test_practitioner_absence_statement_is_bounded_to_its_evidence( + repo_root: Path, relpath: str +) -> None: + """The absence of a practitioner baseline is a fact about one evidence set on one date.""" + text = (repo_root / relpath).read_text(encoding="utf-8") + _assert_practitioner_absence_is_bounded(text) + + for anchor in PRACTITIONER_BOUNDING_LINE_ANCHORS: + assert anchor in text, f"bounding anchor not found for mutation: {anchor}" + removed = text.replace(anchor, "removed") + with pytest.raises(AssertionError, match="practitioner absence statement is unbounded"): + _assert_practitioner_absence_is_bounded(removed) + + for phrase in PRACTITIONER_UNBOUNDED_WORDINGS: + unbounded = text + f"\nNo practitioner study or baseline arm {phrase}.\n" + with pytest.raises(AssertionError, match="asserts an unbounded universal"): + _assert_practitioner_absence_is_bounded(unbounded)