From cc108ea18fe02e24ade3dc1740e3d731c90471c4 Mon Sep 17 00:00:00 2001 From: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> Date: Sat, 5 Sep 2026 00:23:06 -0700 Subject: [PATCH 1/7] conformance(operator-surface): Tier-A kit -- 10 machine checks, 22 bad halves, wired into make test + CI Freeze 1 asks for a Tier-A kit at a named path that runs on every pull request; Freeze 4 asks that every Conformance fixture's bad half has been demonstrated by running it. Both now hold. tests/conformance/operator_surface/ test_tier_a.py one test per Tier-A **Machine check:** name the contract states -- hero.velocity_and_counts, state.not_colour_only, visual.single_source, reads.never_write, calm.keeps_slot, deps.no_framework, antigoals.enforced, push.alarm_only, palette.status_hue_set, perception.floors -- each a pure check function with a GOOD half against the real artifact and at least one BAD half against a deliberately wrong one. Plus two coverage tripwires: one reads the contract and fails if a Tier-A check is unimplemented or stale, one fails if any check ships no bad half. conftest.py CALM / ALARM / EMPTY rendered fixtures, each in its OWN workspace root -- L0 aggregates every project in a workspace, so sharing one root would make the calm render alarming and the empty render non-empty. Six good halves are xfail(strict=True) naming their open ledger row -- never skipped, never deleted. strict is load-bearing: when the fix lands the test XPASSes and the run fails, which is the instruction to flip the row and delete the marker in the same change. Core 4's census and exemption register are IMPORTED from ledger/ rather than reimplemented (OSV1-025's own warning: two censuses disagree silently). The register is hoisted to test_operator_rows.EXEMPTION_REGISTER so the kit can read it, and the kit asserts it still agrees with the live census. Ledger, re-derived from the kit's real results rather than from a file appearing: OSV1-004 GAP -> CONFORMS Core 3 measured over rendered L0/L1/L2: zero status-bearing elements without text or an accessible name. OSV1-012 GAP -> VIOLATION Core 8 measured: slots all survive, but three regions keep their slot and say nothing. OSV1-027 GAP -> CONFORMS Freeze 1, both halves: the kit exists AND runs (Makefile test-conformance-a, CI Tier 6). OSV1-021/-024/-025/-026 stay GAP -- fixtures exist and discriminate, good halves deferred against OSV1-001/-005/-012 and the Tier-B kit. OSV1-030/-031 (out of lane scope, minimal): Freeze 4 retargeted to the Tier-B half; Freeze 5's red-Core tally 10 -> 9. Every retargeted probe carries a mutation that flips it red: the spent "the Tier-A kit file appears" counterfactual is replaced by what must actually move -- the Core row a deferred good half waits on closing, CI dropping the step, or a renderer growing its empty branch. make ledger-mutate: proven 52 / 52. --- .github/workflows/ci.yml | 9 + Makefile | 18 +- ledger/checks/mutation_harness.py | 95 +- ledger/checks/test_operator_rows.py | 313 +++- ledger/rows.yaml | 377 +++-- tests/conformance/__init__.py | 8 + .../conformance/operator_surface/__init__.py | 7 + .../conformance/operator_surface/conftest.py | 218 +++ .../operator_surface/test_tier_a.py | 1360 +++++++++++++++++ 9 files changed, 2185 insertions(+), 220 deletions(-) create mode 100644 tests/conformance/__init__.py create mode 100644 tests/conformance/operator_surface/__init__.py create mode 100644 tests/conformance/operator_surface/conftest.py create mode 100644 tests/conformance/operator_surface/test_tier_a.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index dfec3ee..058b76e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -104,6 +104,15 @@ jobs: - name: Tier 4 -- conformance ledger run: .venv/bin/python -m pytest ledger/checks -v + # Tier 6 -- contracts/operator-surface.v1.md's Tier-A conformance kit. + # Freeze 1 requires it to exist AND to run on every pull request, so it + # is a named step of its own: a kit that runs in no gate is the exact + # failure ledger row CCV1-022 already recorded once. Runs AFTER the + # tiers above because its rendered checks need the same real `bd` and + # dolt those tiers install. + - name: Tier 6 -- operator-surface conformance (Tier A) + run: .venv/bin/python -m pytest tests/conformance/operator_surface -v + # Tier 5 -- the amplifier tool module's own suite # (modules/tool-work-tracker/tests). The only place the post-reclaim # custody behaviour of the AGENT SEAM (work_claim / work_declare / diff --git a/Makefile b/Makefile index 019ac82..18546f2 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: venv test test-unit test-integration test-cli test-ledger ledger-mutate test-module check lint types doctor clean +.PHONY: venv test test-unit test-integration test-cli test-ledger ledger-mutate test-conformance-a test-module check lint types doctor clean PYTHON ?= python3.12 VENV := .venv @@ -52,6 +52,15 @@ test-ledger: ledger-mutate: $(PY) -m ledger.checks.mutation_harness +## Tier 6 -- operator-surface conformance, Tier A: the machine checks +## contracts/operator-surface.v1.md names for its own Tier A, each as a +## discriminating good/bad pair (Freeze 1 requires this kit to exist AND to +## run on every pull request; Freeze 4 requires every bad half to have been +## demonstrated). Its static checks need nothing; its RENDERED checks stand up +## the isolated dolt server and skip loudly without a real `bd`. +test-conformance-a: + $(PYTEST) tests/conformance/operator_surface -v + ## Tier 5 -- tool module: modules/tool-work-tracker's own suite, the only ## place the post-reclaim custody behaviour of the AGENT SEAM (work_claim / ## work_declare / work_resolve / work_release) is asserted mechanically. @@ -65,7 +74,7 @@ ledger-mutate: test-module: $(PYTEST) modules/tool-work-tracker/tests -v -## All five tiers. Two pytest invocations (see `test-module` above), and +## Every tier. Two pytest invocations (see `test-module` above), and ## deliberately NOT fail-fast between them: the whole point of wiring the ## module suite in (ledger row CCV1-022) is that it stops being silently ## skippable, and a pre-existing failure in the root suite must not go back @@ -77,6 +86,11 @@ test: $(PYTEST) modules/tool-work-tracker/tests -v || rc=$$?; \ exit $$rc +## `test` above already COLLECTS tests/conformance (it is under `tests`), so +## this is the named, separately-runnable entry point rather than a second +## invocation -- running the kit twice in one `make test` would double its +## fixture cost and prove nothing new. + ## Lint + type-check. check: lint types diff --git a/ledger/checks/mutation_harness.py b/ledger/checks/mutation_harness.py index 505c75f..6bc9983 100644 --- a/ledger/checks/mutation_harness.py +++ b/ledger/checks/mutation_harness.py @@ -364,13 +364,19 @@ def _m023_test_location_regresses(w: World) -> None: # shape and must go red the moment the shape is right. The seven green rows get # the known-wrong shape they forbid instead (direction REGRESSION). # -# Eleven rows pin the absence of a conformance-kit file, and their +# Six rows still pin the absence of the TIER-B kit file, and their # counterfactual is `world.touch(...)` -- the file simply existing. That is a # WEAKER mutation than the source-shape ones, deliberately and visibly: the # thing being proven is only that the pin notices the path appearing, which is -# exactly what those rows claim and no more. Four of them carry a SECOND +# exactly what those rows claim and no more. Three of them carry a SECOND # mutation against their substantive half, so the weak half is never the only # evidence. +# +# The TIER-A kit now EXISTS (OSV1-027), so "the file appears" is spent as a +# counterfactual for the rows that used it. Each has been retargeted at what +# actually has to move for it to go green: for the three Conformance-fixture +# rows, the Core row their deferred good half waits on closing; for Freeze 1, +# CI dropping the step that runs the kit. # ============================================================================= TIER_A_KIT = op_probes.TIER_A_KIT @@ -501,9 +507,20 @@ def _mo011_a_second_motion_block_appears(w: World) -> None: ) -def _mo012_the_empty_sentence_becomes_a_zero(w: World) -> None: - """REGRESSION: the calm queue is celebrated as a numeral instead of said.""" - w.replace(WEBAPP, "Nothing is waiting to be claimed in this queue right now.", "0") +def _mo012_the_empty_slot_gains_its_sentence(w: World) -> None: + """FIXED: `render_attention_queue` grows the empty branch Core 8 asks for, + so the L0 region that keeps its slot finally says so.""" + w.replace( + WIDGETS, + " rows_html: list[str] = []\n" + ' for r in data["rows"]:\n' + ' priority_label = _esc(r["priority"].upper())', + ' if not data["rows"]:\n' + " return '
Nothing needs you right now.
'\n" + " rows_html: list[str] = []\n" + ' for r in data["rows"]:\n' + ' priority_label = _esc(r["priority"].upper())', + ) def _mo013_a_template_engine_is_declared(w: World) -> None: @@ -538,10 +555,6 @@ def _mo017_a_second_push_call_site_appears(w: World) -> None: w.append(WEBAPP, "\ndef _ledger_mutation():\n WP.fire_reclaim_alarm(1, 2, 3)\n") -def _mo_tier_a_kit_appears(w: World) -> None: - w.touch(TIER_A_KIT) - - def _mo_tier_b_kit_appears(w: World) -> None: w.touch(TIER_B_KIT) @@ -569,9 +582,16 @@ def _mo023_a_swept_breakpoint_disappears(w: World) -> None: w.replace(WEBTHEME, "@media (max-width:1280px){", "@media (max-width:1281px){") -def _mo027_the_makefile_wires_the_kit(w: World) -> None: - """The wiring half of Freeze 1: existing is not the same as running.""" - w.append(MAKEFILE, "\ntest-conformance:\n\t$(PYTEST) tests/conformance -v\n") +def _mo027_ci_stops_running_the_kit(w: World) -> None: + """REGRESSION on the wiring half of Freeze 1: the kit still exists, but CI + no longer runs it -- which is exactly the "green claims nobody executed" + failure CCV1-022 already recorded once in this repo.""" + w.replace( + CI_WORKFLOW, + " - name: Tier 6 -- operator-surface conformance (Tier A)\n" + " run: .venv/bin/python -m pytest tests/conformance/operator_surface -v\n", + "", + ) def _mo029_an_artifact_directory_appears(w: World) -> None: @@ -591,6 +611,26 @@ def _mo031_a_red_core_row_goes_green(w: World) -> None: ) +def _mo025_the_literal_style_row_goes_green(w: World) -> None: + """FIXED: OSV1-005 closes, so Conformance 6's deferred good half should now + pass -- and this row must be re-derived from the PASSING pair.""" + w.replace( + ROWS_PATH, + " disposition: VIOLATION\n work: work_item_pipeline-np3", + " disposition: CONFORMS\n work: work_item_pipeline-np3", + ) + + +def _mo026_the_empty_slot_row_goes_green(w: World) -> None: + """FIXED: OSV1-012 closes, so Conformance 7's deferred good halves should + now pass.""" + w.replace( + ROWS_PATH, + " disposition: VIOLATION\n work: work_item_pipeline-c1a", + " disposition: CONFORMS\n work: work_item_pipeline-c1a", + ) + + def _mo032_the_register_grows(w: World) -> None: """Freeze 6's enumerated half moves. NOTE, honestly: the OTHER half (zero literal sites remaining) is not simulable in memory -- it would mean @@ -765,8 +805,8 @@ def _mo034_the_changelog_records_a_look(w: World) -> None: ), Mutation( "OSV1-012", - "the calm queue's empty sentence becomes a bare zero", - _mo012_the_empty_sentence_becomes_a_zero, + "the empty attention queue grows the sentence Core 8 asks for", + _mo012_the_empty_slot_gains_its_sentence, ), Mutation( "OSV1-013", "the manifest declares a template engine", _mo013_a_template_engine_is_declared @@ -790,7 +830,7 @@ def _mo034_the_changelog_records_a_look(w: World) -> None: _mo017_a_second_push_call_site_appears, ), Mutation("OSV1-020", "the Tier-B kit file appears", _mo_tier_b_kit_appears), - Mutation("OSV1-021", "the Tier-A kit file appears", _mo_tier_a_kit_appears), + Mutation("OSV1-021", "the Tier-B kit file appears", _mo_tier_b_kit_appears), Mutation("OSV1-022", "the Tier-B kit file appears", _mo_tier_b_kit_appears), Mutation( "OSV1-022", @@ -803,14 +843,25 @@ def _mo034_the_changelog_records_a_look(w: World) -> None: "a swept breakpoint disappears from the stylesheet", _mo023_a_swept_breakpoint_disappears, ), - Mutation("OSV1-024", "the Tier-A kit file appears", _mo_tier_a_kit_appears), - Mutation("OSV1-025", "the Tier-A kit file appears", _mo_tier_a_kit_appears), - Mutation("OSV1-026", "the Tier-A kit file appears", _mo_tier_a_kit_appears), - Mutation("OSV1-027", "the Tier-A kit file appears", _mo_tier_a_kit_appears), + Mutation( + "OSV1-024", + "OSV1-001 closes, so Conformance 5's deferred good half should now pass", + _mo031_a_red_core_row_goes_green, + ), + Mutation( + "OSV1-025", + "OSV1-005 closes, so Conformance 6's deferred good half should now pass", + _mo025_the_literal_style_row_goes_green, + ), + Mutation( + "OSV1-026", + "OSV1-012 closes, so Conformance 7's deferred good halves should now pass", + _mo026_the_empty_slot_row_goes_green, + ), Mutation( "OSV1-027", - "the Makefile wires a conformance target (the 'runs in a gate' half)", - _mo027_the_makefile_wires_the_kit, + "CI stops running the Tier-A kit (the 'runs in a gate' half)", + _mo027_ci_stops_running_the_kit, ), Mutation("OSV1-028", "the Tier-B kit file appears", _mo_tier_b_kit_appears), Mutation("OSV1-029", "the Tier-B kit file appears", _mo_tier_b_kit_appears), @@ -819,7 +870,7 @@ def _mo034_the_changelog_records_a_look(w: World) -> None: "a Tier-B artifact directory appears", _mo029_an_artifact_directory_appears, ), - Mutation("OSV1-030", "the Tier-A kit file appears", _mo_tier_a_kit_appears), + Mutation("OSV1-030", "the Tier-B kit file appears", _mo_tier_b_kit_appears), Mutation( "OSV1-031", "one of the ten red Core-carrying rows flips to CONFORMS", diff --git a/ledger/checks/test_operator_rows.py b/ledger/checks/test_operator_rows.py index b6fe543..617369f 100644 --- a/ledger/checks/test_operator_rows.py +++ b/ledger/checks/test_operator_rows.py @@ -41,12 +41,14 @@ from __future__ import annotations +import ast import re from ._support import ( CHARTSVG, LITERAL, OPERATOR_CONTRACT_PATH, + PINNING_DISPOSITIONS, PYPROJECT, REPO_ROOT, SUPERVISOR, @@ -92,6 +94,66 @@ def _exists(rel: str) -> bool: return (REPO_ROOT / rel).exists() +# --------------------------------------------------------------------------- +# Reading the Tier-A conformance kit. +# +# Several rows below are about the KIT rather than about `src/` -- whether the +# check the contract names is implemented, whether it ships a bad half, and +# whether its good half is still DEFERRED against an open row. All three go +# through `read()` so the mutation harness's reader injection reaches them. +# --------------------------------------------------------------------------- + + +def _kit_source() -> str: + return read(REPO_ROOT / TIER_A_KIT) + + +def _kit_defs(kit: str) -> frozenset[str]: + return frozenset( + n.name + for n in ast.walk(ast.parse(kit)) + if isinstance(n, ast.FunctionDef | ast.AsyncFunctionDef) + ) + + +def _kit_bad_halves(kit: str, good: str) -> list[str]: + """The bad-half tests paired with a check's good half, by name.""" + return sorted(n for n in _kit_defs(kit) if n.startswith(f"{good}_bad_half_")) + + +def _kit_deferred_rows(kit: str, test_name: str) -> frozenset[str]: + """Row ids named in `test_name`'s `xfail` reason -- empty if not deferred. + + A kit test marked `xfail(strict=True)` is a check that RUNS and currently + FAILS, with the ledger row that owns the failure named in its reason. That + marker is what a red Conformance row pins: when the product fix lands the + test XPASSes, the run fails, and the marker and the row move together. + """ + for node in ast.walk(ast.parse(kit)): + if not isinstance(node, ast.FunctionDef) or node.name != test_name: + continue + for dec in node.decorator_list: + if not isinstance(dec, ast.Call): + continue + if not ast.unparse(dec.func).endswith("xfail"): + continue + for kw in dec.keywords: + if kw.arg == "reason" and isinstance(kw.value, ast.Constant): + return frozenset(re.findall(r"OSV1-\d{3}", str(kw.value.value))) + return frozenset() + return frozenset() + + +def _widgets_function(name: str) -> str: + """One function's source out of `widgets.py`, via the patched reader.""" + src = read(WIDGETS) + lines = src.splitlines() + for node in ast.walk(ast.parse(src)): + if isinstance(node, ast.FunctionDef) and node.name == name: + return "\n".join(lines[node.lineno - 1 : (node.end_lineno or node.lineno)]) + raise AssertionError(f"widgets.py no longer defines {name!r}") + + def _repo_mentions(needle: str) -> bool: """Does any tracked source/config file name this string? @@ -259,12 +321,28 @@ def test_row_osv1_003() -> None: def test_row_osv1_004() -> None: - """Core 3 GAP pin: no rendered-fixture kit, and the chip vocabulary that - the kit will check is still the five-word map measured at seed. + """Core 3 CONFORMS: the rendered check exists, is not deferred, and the + chip vocabulary it walks still gives every status a WORD. + + This row rests on the kit's real result over rendered L0/L1/L2 (recorded + in the row itself), which no in-process probe can re-run. So this probe + asserts the two things that CAN still be seen going wrong from here: the + check disappearing or being deferred, and a status losing its word in the + map the check walks. """ - assert not _exists(TIER_A_KIT), ( - f"OSV1-004 (Core 3): {TIER_A_KIT} now exists -- re-derive this row from the " - f"kit's real result over rendered L0/L1/L2 fixtures (work_item_pipeline-c1a)." + assert _exists(TIER_A_KIT), ( + f"OSV1-004 (Core 3) REGRESSION: {TIER_A_KIT} is gone. Core 3's check is Tier " + f"A and scoped to RENDERED fixtures -- without the kit nothing asserts it, and " + f"this row has no basis to be green." + ) + kit = _kit_source() + assert {"check_state_not_colour_only", "test_state_not_colour_only"} <= _kit_defs(kit), ( + "OSV1-004 (Core 3) REGRESSION: the kit no longer implements " + "`state.not_colour_only` over rendered pages." + ) + assert not _kit_deferred_rows(kit, "test_state_not_colour_only"), ( + "OSV1-004 (Core 3) REGRESSION: the rendered Core 3 check is now marked xfail. " + "This row is CONFORMS because the check PASSES; a deferred check is a red row." ) browse = read(WEBBROWSE) labels = set(re.findall(r'"(?:open|held|blocked|deferred|resolved)":\s*"([A-Z]+)"', browse)) @@ -366,15 +444,15 @@ def test_row_osv1_005() -> None: # --------------------------------------------------------------- OSV1-006 -def test_row_osv1_006() -> None: - """Core 4 CONFORMS: computed-geometry sites are EXACTLY the register. - - This is the exemption register itself (Phase-1 ruling Need 2 put it in - `ledger/` so shrinking it needs no amendment). Shrinking passes only after - the register below is shrunk to match; GROWING fails immediately, which is - the direction that matters. - """ - register = { +#: THE exemption register Core 4 names -- "The register lives in `ledger/`, not +#: in this contract, so that shrinking it is a convergent change requiring no +#: amendment." Module-level rather than local to `test_row_osv1_006` below so +#: that the Tier-A conformance kit +#: (`tests/conformance/operator_surface/test_tier_a.py`, Conformance 6) reads +#: THIS register rather than growing a second copy: OSV1-025's own notes name +#: that as the failure to avoid, because two censuses disagree silently. +EXEMPTION_REGISTER: frozenset[str] = frozenset( + { "chartsvg.py:268", "chartsvg.py:464", "chartsvg.py:488", @@ -399,6 +477,18 @@ def test_row_osv1_006() -> None: "widgets.py:834", "widgets.py:1110", } +) + + +def test_row_osv1_006() -> None: + """Core 4 CONFORMS: computed-geometry sites are EXACTLY the register. + + This is the exemption register itself (Phase-1 ruling Need 2 put it in + `ledger/` so shrinking it needs no amendment). Shrinking passes only after + `EXEMPTION_REGISTER` above is shrunk to match; GROWING fails immediately, + which is the direction that matters. + """ + register = EXEMPTION_REGISTER observed = set(style_sites_in("COMPUTED")) unregistered = observed - register assert not unregistered, ( @@ -587,13 +677,32 @@ def test_row_osv1_011() -> None: def test_row_osv1_012() -> None: - """Core 8 GAP pin: no two-render kit, and the empty sentences measured at - seed are still the ones the kit will look for. + """Core 8 VIOLATION pin: two widget renderers still have no empty branch, + and the kit's two `calm.keeps_slot` halves are still deferred against this + row. + + Pinned on the RENDERERS rather than on a rendered page, because that is + what an in-process probe can see: `render_attention_queue` and + `render_agents_panel` return their container unconditionally, so an empty + one is a slot with nothing in it. Giving either an empty branch flips this + pin -- which is the fix landing. """ - assert not _exists(TIER_A_KIT), ( - f"OSV1-012 (Core 8): {TIER_A_KIT} now exists -- re-derive this row from a real " - f"empty-vs-populated render comparison (work_item_pipeline-c1a)." - ) + for func in ("render_attention_queue", "render_agents_panel"): + body = _widgets_function(func) + assert "if not data[" not in body, ( + f"OSV1-012 (Core 8) PIN BROKE THE RIGHT WAY: `{func}` now has an empty " + f"branch. If it emits the empty SENTENCE Core 8 requires, re-run the " + f"Tier-A kit's `calm.keeps_slot` halves, flip OSV1-012 to CONFORMS, delete " + f"their xfail markers and retarget this probe -- all in the same change " + f"(work_item_pipeline-c1a)." + ) + kit = _kit_source() + for test_name in ("test_calm_keeps_slot", "test_calm_keeps_slot_l1"): + assert "OSV1-012" in _kit_deferred_rows(kit, test_name), ( + f"OSV1-012 (Core 8) PIN BROKE THE RIGHT WAY: the kit's `{test_name}` is no " + f"longer deferred against this row. A passing good half is the fix -- flip " + f"the row in the same change." + ) assert contains(WIDGETS, '"All clear"'), ( "OSV1-012 (Core 8): the calm headline 'All clear' is gone. Calm must stay " "STATED -- and must not become a triumphant zero." @@ -829,12 +938,22 @@ def test_row_osv1_020() -> None: def test_row_osv1_021() -> None: - """Conformance 2 pin: NEITHER named kit exists (this fixture spans tiers).""" - assert not _exists(TIER_A_KIT) and not _exists(TIER_B_KIT), ( - "OSV1-021 (Conformance 2): a named kit path appeared. This is the one fixture " - "that spans BOTH tiers -- it goes green only when the Tier-A accessible-name " - "half (work_item_pipeline-c1a) AND the Tier-B hue half " - "(work_item_pipeline-qgo) both land and both discriminate." + """Conformance 2 pin: the Tier-A half landed; the Tier-B hue half has not, + and that absence is what keeps this row red.""" + assert not _exists(TIER_B_KIT), ( + f"OSV1-021 (Conformance 2): {TIER_B_KIT} now exists. This is the one fixture " + f"that spans BOTH tiers -- the Tier-A accessible-name half already landed " + f"(work_item_pipeline-c1a, measured in OSV1-004); re-derive this row from the " + f"Tier-B hue half's own demonstrated pair (work_item_pipeline-qgo)." + ) + kit = _kit_source() + assert "check_state_not_colour_only" in _kit_defs(kit), ( + "OSV1-021 (Conformance 2): the Tier-A half is gone from the kit. This row is " + "red on the Tier-B half ONLY -- losing the Tier-A half is a second, new defect." + ) + assert _kit_bad_halves(kit, "test_state_not_colour_only"), ( + "OSV1-021 (Conformance 2): the Tier-A half no longer ships a bad half. A " + "fixture whose bad half nobody runs is a claim, not a fixture (Freeze 4)." ) assert contains( OPERATOR_CONTRACT_PATH, "fixture whose status chips carry only a status class" @@ -871,10 +990,27 @@ def test_row_osv1_023() -> None: def test_row_osv1_024() -> None: - """Conformance 5 pin: no Tier-A kit, so hero composition is unasserted.""" - assert not _exists(TIER_A_KIT), ( - f"OSV1-024 (Conformance 5): {TIER_A_KIT} now exists -- re-derive from the " - f"demonstrated good/bad pair (work_item_pipeline-c1a)." + """Conformance 5 pin: the fixture exists and discriminates, and its GOOD + half is still deferred against OSV1-001.""" + kit = _kit_source() + assert "check_hero_velocity_and_counts" in _kit_defs(kit), ( + f"OSV1-024 (Conformance 5): {TIER_A_KIT} no longer implements " + f"`hero.velocity_and_counts` at the location the contract names." + ) + assert _kit_bad_halves(kit, "test_hero_velocity_and_counts"), ( + "OSV1-024 (Conformance 5): the fixture no longer ships a bad half -- a fixture " + "that cannot be watched failing is a claim (Freeze 4)." + ) + assert "OSV1-001" in _kit_deferred_rows(kit, "test_hero_velocity_and_counts"), ( + "OSV1-024 (Conformance 5) PIN BROKE THE RIGHT WAY: the good half is no longer " + "deferred against OSV1-001. If the hero now carries velocity with its window " + "and all four counts, flip OSV1-001 AND this row and retarget both probes in " + "the same change (work_item_pipeline-ujy)." + ) + assert row("OSV1-001")["disposition"] in PINNING_DISPOSITIONS, ( + "OSV1-024 (Conformance 5) PIN BROKE THE RIGHT WAY: OSV1-001 is no longer red, " + "so Conformance 5's good half should now pass against a conforming hero. " + "Re-derive this row from the PASSING pair, never from the fixture's presence." ) assert contains( OPERATOR_CONTRACT_PATH, @@ -883,11 +1019,29 @@ def test_row_osv1_024() -> None: def test_row_osv1_025() -> None: - """Conformance 6 pin: no Tier-A kit, so the register is checked only here.""" - assert not _exists(TIER_A_KIT), ( - f"OSV1-025 (Conformance 6): {TIER_A_KIT} now exists. Make sure it reads THIS " - f"ledger's register (OSV1-006) rather than growing its own copy -- two censuses " - f"will disagree, silently (work_item_pipeline-c1a)." + """Conformance 6 pin: the fixture exists, reads THIS ledger's register, and + its GOOD half is still deferred against OSV1-005.""" + kit = _kit_source() + assert "check_visual_single_source" in _kit_defs(kit), ( + f"OSV1-025 (Conformance 6): {TIER_A_KIT} no longer implements " + f"`visual.single_source` at the location the contract names." + ) + assert _kit_bad_halves(kit, "test_visual_single_source"), ( + "OSV1-025 (Conformance 6): the fixture no longer ships a bad half (Freeze 4)." + ) + assert "EXEMPTION_REGISTER" in kit, ( + "OSV1-025 (Conformance 6): the kit stopped importing this ledger's exemption " + "register. ONE census, ONE register (Phase-1 ruling Need 2) -- a second copy " + "disagrees with this one, silently." + ) + assert "OSV1-005" in _kit_deferred_rows(kit, "test_visual_single_source"), ( + "OSV1-025 (Conformance 6) PIN BROKE THE RIGHT WAY: the good half is no longer " + "deferred against OSV1-005. Flip OSV1-005 AND this row and retarget both " + "probes in the same change (work_item_pipeline-np3)." + ) + assert row("OSV1-005")["disposition"] in PINNING_DISPOSITIONS, ( + "OSV1-025 (Conformance 6) PIN BROKE THE RIGHT WAY: OSV1-005 is no longer red, " + "so Conformance 6's good half should now pass. Re-derive from the PASSING pair." ) assert contains(OPERATOR_CONTRACT_PATH, 'style="color:#D9A253"'), ( "OSV1-025: Conformance 6's named bad specimen moved in the contract -- re-review the row." @@ -895,10 +1049,24 @@ def test_row_osv1_025() -> None: def test_row_osv1_026() -> None: - """Conformance 7 pin: no Tier-A kit, so empty-vs-populated is uncompared.""" - assert not _exists(TIER_A_KIT), ( - f"OSV1-026 (Conformance 7): {TIER_A_KIT} now exists -- re-derive from a real " - f"two-render comparison (work_item_pipeline-c1a)." + """Conformance 7 pin: the two-render fixture exists and discriminates, and + its GOOD halves are still deferred against OSV1-012.""" + kit = _kit_source() + assert "check_calm_keeps_slot" in _kit_defs(kit), ( + f"OSV1-026 (Conformance 7): {TIER_A_KIT} no longer implements " + f"`calm.keeps_slot` at the location the contract names." + ) + assert _kit_bad_halves(kit, "test_calm_keeps_slot"), ( + "OSV1-026 (Conformance 7): the fixture no longer ships a bad half (Freeze 4)." + ) + assert "OSV1-012" in _kit_deferred_rows(kit, "test_calm_keeps_slot"), ( + "OSV1-026 (Conformance 7) PIN BROKE THE RIGHT WAY: the good half is no longer " + "deferred against OSV1-012. Flip OSV1-012 AND this row and retarget both " + "probes in the same change (work_item_pipeline-c1a)." + ) + assert row("OSV1-012")["disposition"] in PINNING_DISPOSITIONS, ( + "OSV1-026 (Conformance 7) PIN BROKE THE RIGHT WAY: OSV1-012 is no longer red, " + "so Conformance 7's good halves should now pass. Re-derive from the PASSING pair." ) assert contains( OPERATOR_CONTRACT_PATH, "a render that drops empty widgets, or renders a hero-scale `0`" @@ -910,27 +1078,38 @@ def test_row_osv1_026() -> None: def test_row_osv1_027() -> None: - """Freeze 1 pin: the Tier-A kit neither exists nor runs in any gate. + """Freeze 1 CONFORMS: the Tier-A kit exists AND runs in a real gate. - Both halves are pinned, because a kit that exists but runs in nothing is + Both halves are asserted, because a kit that exists but runs in nothing is the exact failure this repo already measured once (CCV1-022: a whole suite - of green claims nobody had ever executed). + of green claims nobody had ever executed). Matched on the KIT PATH, never + on the word "conformance": the Makefile and ci.yml already say "conformance + ledger" about Tier 4, and a check a pre-existing comment satisfies asserts + nothing. """ - assert not _exists(TIER_A_KIT), ( - f"OSV1-027 (Freeze 1): {TIER_A_KIT} now exists. Freeze 1 ALSO requires it to " - f"run on every pull request -- do not flip this row on the file alone " - f"(work_item_pipeline-c1a)." - ) - # Matched on the KIT PATH, never on the word "conformance": the Makefile - # and ci.yml already say "conformance ledger" about Tier 4, and a probe - # that a pre-existing comment satisfies is a probe asserting nothing. - assert "tests/conformance" not in read(MAKEFILE), ( - "OSV1-027 (Freeze 1): the Makefile now has a target covering tests/conformance " - "-- the wiring half is landing. Re-derive." - ) - assert "tests/conformance" not in read(CI_WORKFLOW), ( - "OSV1-027 (Freeze 1): ci.yml now runs tests/conformance -- the 'runs on every " - "pull request' half is landing. Re-derive." + assert _exists(TIER_A_KIT), f"OSV1-027 (Freeze 1) REGRESSION: {TIER_A_KIT} is gone." + make = read(MAKEFILE) + assert "tests/conformance" in make and "test-conformance-a:" in make, ( + "OSV1-027 (Freeze 1) REGRESSION: the Makefile no longer carries a target " + "covering tests/conformance. Existing is not the same as running." + ) + ci = read(CI_WORKFLOW) + assert "tests/conformance/operator_surface" in ci, ( + "OSV1-027 (Freeze 1) REGRESSION: ci.yml no longer runs the Tier-A kit -- " + "Freeze 1's second half is 'runs on every pull request'." + ) + assert "pull_request" in ci, ( + "OSV1-027 (Freeze 1) REGRESSION: the workflow that runs the kit no longer " + "triggers on pull_request." + ) + kit = _kit_source() + assert "TIER_A_CHECKS" in kit and _kit_defs(kit) >= { + "test_every_tier_a_machine_check_the_contract_names_is_implemented_here", + "test_every_check_ships_a_bad_half", + }, ( + "OSV1-027 (Freeze 1) REGRESSION: the kit dropped its own coverage tripwires. " + "Without them it can quietly cover less of the contract than it claims, which " + "is the failure Freeze 1 and Freeze 4 exist to prevent." ) @@ -969,11 +1148,17 @@ def test_row_osv1_029() -> None: def test_row_osv1_030() -> None: - """Freeze 4 pin: neither kit exists, so no fixture has been demonstrated.""" - assert not _exists(TIER_A_KIT) and not _exists(TIER_B_KIT), ( - "OSV1-030 (Freeze 4): a kit path appeared. 'Demonstrated by running it' is the " - "whole clause -- record WHICH revert produced WHICH failure, the way " - "CCV1-023 did for the custody family, before flipping this row." + """Freeze 4 pin: the Tier-A fixtures are demonstrated; no browser kit + exists, so the three Tier-B fixtures are not.""" + assert not _exists(TIER_B_KIT), ( + "OSV1-030 (Freeze 4): the Tier-B kit path appeared. 'Demonstrated by running " + "it' is the whole clause -- record WHICH revert produced WHICH failure, the " + "way CCV1-023 did for the custody family, before flipping this row." + ) + assert _exists(TIER_A_KIT) and _kit_bad_halves(_kit_source(), "test_calm_keeps_slot"), ( + "OSV1-030 (Freeze 4): the Tier-A fixtures' demonstrated bad halves are gone. " + "This row is red on the TIER-B half only -- losing the Tier-A half is a " + "second, new defect." ) @@ -1002,8 +1187,8 @@ def test_row_osv1_031() -> None: "to CONFORMS and retarget this probe to assert no Core row is red " "(work_item_pipeline-umm)." ) - assert len(red) == 10, ( - f"OSV1-031 (Freeze 5): pinned 10 red Core-carrying rows, observed {len(red)}: " + assert len(red) == 9, ( + f"OSV1-031 (Freeze 5): pinned 9 red Core-carrying rows, observed {len(red)}: " f"{red}. Movement in either direction means this gate's tally changed -- update " f"the pin and the row's notes in the same change." ) diff --git a/ledger/rows.yaml b/ledger/rows.yaml index bd3fa2b..3c9b3b7 100644 --- a/ledger/rows.yaml +++ b/ledger/rows.yaml @@ -1059,43 +1059,56 @@ presence so the sweep, when built, has a real target. - id: OSV1-004 - title: state is never colour-only -- unmeasured on a rendered page, no kit + title: state is never colour-only -- measured on rendered L0/L1/L2, and it holds contract: file: contracts/operator-surface.v1.md clause: Core 3 quote: | Every element carrying a state carries it in text as well as hue — a status chip has a word, not only a class. - disposition: GAP - work: work_item_pipeline-c1a + disposition: CONFORMS assertion: kind: probe ref: test_row_osv1_004 notes: > - PINNING ROW -- the probe asserts the CURRENT, KNOWN-WRONG shape on purpose; - a passing probe here is NOT conformance. Flip direction VIOLATION-MOVEMENT. - - GAP, REASON "KIT NOT BUILT". Core 3's check (`state.not_colour_only`) is - Tier A, but the contract scopes it to "the rendered L0/L1/L2 fixtures", and - `tests/conformance/operator_surface/test_tier_a.py` does not exist. This - ledger kit is in-process only (no bd, no dolt, no subprocess), so it can - assert the SOURCE shape of the chip vocabulary and nothing about a page. - - WHAT THE SOURCE SHAPE ALREADY SHOWS, measured 2026-09-04 -- encouraging, - and deliberately NOT treated as conformance: - webbrowse.py:139-152 every item status maps to BOTH a class - (`_ITEM_STATUS_CHIP_CLASS`) and a WORD - (`_ITEM_STATUS_CHIP_LABEL`: READY / HELD / BLOCKED - / DEFERRED / RESOLVED) -- five statuses, five words. - webbrowse.py:849-851 the L2 chip renders `item.status.upper()`. - widgets.py:1162,1164 legend dots carry adjacent label text in the same - `.li` span. - KNOWN CANDIDATES THE RENDERED CHECK MUST ADJUDICATE, which no source probe - can settle: `webapp.py:548` `` (an - attribute-only name), `webbrowse.py:119` a bare `` - prefixed to the blocked breakdown row, and `webapp.py:839` - `` (in `_sidebar_html`, which measured as dead - chrome -- no route passes `sidebar_html`). + RE-DERIVED 2026-09-05 from a REAL RENDERED MEASUREMENT -- never from the + fact that a kit file appeared. + `tests/conformance/operator_surface/test_tier_a.py` now implements Core 3's + `state.not_colour_only` and runs it over rendered L0, L1 and L2 pages of + the ALARM fixture (one item held past custody TTL, one blocked -- which is + Conformance 2's own scenario, so every status chip this surface can render + is actually on the page). Result: ZERO status-bearing elements without text + or an accessible name, on all three levels. + + WHAT THE CHECK COUNTS AS STATUS-BEARING, so the scope of this green is + legible rather than implied: a class that IS a status word, or a status + word under one of this surface's status-class prefixes (`st-`, `is-`, + `sev-`, `tab-`, `status-`). The vocabulary is read out of the app's OWN + maps -- `webbrowse._ITEM_STATUS_CHIP_LABEL`, `widgets._HERO_ICON`, the + three status hue tokens -- rather than written down in the kit, so a status + the app grows cannot be silently unchecked. A decorative, unlabelled mark + INSIDE a status-bearing element that is itself named is excused: the state + is already carried in that ancestor's words, which is exactly the clause. + + THE THREE CANDIDATES THE SEED ROW LEFT FOR THE RENDERED CHECK TO + ADJUDICATE, now adjudicated: `webapp.py:548`'s + `` carries an accessible name; + `webbrowse.py:119`'s bare `` sits inside + `Blocked N`, which carries the word; + `webapp.py:839`'s `` never reaches a rendered + page at all (dead chrome -- no route passes `sidebar_html`). + + DISCRIMINATING PAIR DEMONSTRATED (Freeze 4). The bad half is Conformance + 2's own specimen -- `` -- and the same + function reports it + (`test_state_not_colour_only_bad_half_chip_with_only_a_class`). Two further + bad halves keep the check from degenerating into "has text": an + `aria-label`-only chip PASSES, and a named ancestor excuses only its own + descendants, never a sibling. + + WHAT THIS ROW STILL DOES NOT COVER: the HUE half of Conformance 2 (the + alarm region's colour is `--alarm` or `--blocked`) is Tier B and belongs to + OSV1-021's second half. - id: OSV1-005 title: >- @@ -1467,7 +1480,9 @@ motion-preference trace in OSV1-010. - id: OSV1-012 - title: calm is reported not celebrated, and empty widgets keep their slot -- no kit + title: >- + two empty widgets keep their slot and say nothing -- measured, not a + missing kit contract: file: contracts/operator-surface.v1.md clause: Core 8 @@ -1475,7 +1490,7 @@ The calm state is stated plainly and never rendered as a triumphant zero. A widget with nothing to show keeps its slot and says so in a sentence, so the page does not reflow between calm and alarm. - disposition: GAP + disposition: VIOLATION work: work_item_pipeline-c1a assertion: kind: probe @@ -1484,27 +1499,40 @@ PINNING ROW -- the probe asserts the CURRENT, KNOWN-WRONG shape on purpose; a passing probe here is NOT conformance. Flip direction VIOLATION-MOVEMENT. - GAP, REASON "KIT NOT BUILT". Core 8's check (`calm.keeps_slot`) is Tier A - but is defined against an ALL-EMPTY FIXTURE compared to a populated one -- - "every widget region present on the populated fixture is present, each - carries its empty sentence, and no numeral renders at hero scale outside - the Core 1 hero". That is a two-render comparison this in-process kit - cannot perform; `tests/conformance/operator_surface/test_tier_a.py` does - not exist. - - WHAT THE SOURCE ALREADY CARRIES, measured 2026-09-04, and NOT credited as - conformance -- the clause's claim is about region PRESENCE and reflow, - which no source probe can see: - widgets.py:423,433 the verdict headline is "All clear" (calm) / - "Idle" -- a plain statement, not a triumphant zero. - webapp.py:2275 "Nothing is waiting to be claimed in this queue - right now." with the figure rendered as an em-dash - (`—` plus - "No ready items"), not a hero-scale `0`. - chartsvg.py:295 "No activity in this window". - webbrowse.py:498-503 "No items match this filter." - Unknown until rendered: whether every widget region survives the empty - render at all, and whether the page reflows between calm and alarm. + WAS GAP "KIT NOT BUILT"; NOW VIOLATION, MEASURED. The Tier-A kit exists and + performs the two-render comparison this clause defines, so the reason for + the red changed from "nobody has looked" to "we looked and it fails". + + MEASURED 2026-09-05, L0 and L1 rendered against an all-empty project and + against the populated ALARM fixture, regions matched by section id and by + panel heading: + SLOTS SURVIVE every widget region on the populated render is present on + the empty one, on BOTH levels. That half of the clause + holds and the check says so. + SENTENCES DO NOT three regions keep their slot and say nothing at all: + L0 "Needs you -- ranked" (`widgets.render_attention_queue` + returns a bare `
` when + `rows` is empty), L1 "Agents on " + (`widgets.render_agents_panel`, the same shape), and L1 + "Status breakdown" (a legend of zeroes, no sentence). + NO HERO-SCALE NUMERAL nothing at or above the hero's own type size + (`--text-display-size`, resolved from the token block -- + the floor is derived from the hero, never written down) + renders a numeral outside `#verdict-hero`. That limb + holds. + + SO THE ROW IS RED ON ONE LIMB OF THREE, and the kit records which: the + two `calm.keeps_slot` tests are `xfail(strict=True)` naming THIS row. When + the three regions gain their sentence the tests XPASS, which FAILS the run + -- that failure is the instruction to flip this row and delete the markers + in the same change. + + WHAT THE SOURCE ALREADY CARRIES, and still does: `widgets.py:423,433` the + verdict headline is "All clear" / "Idle" -- a plain statement, never a + triumphant zero; `webapp.py:2275` the ready queue says "Nothing is waiting + to be claimed in this queue right now." with the figure as an em-dash; + `chartsvg.py:295` "No activity in this window"; `webbrowse.py:498-503` "No + items match this filter." Four widgets get this right; three do not. - id: OSV1-013 title: no front-end framework, bundler, template engine, or build step @@ -1788,7 +1816,9 @@ (webpwa.py:121-122, webtrust.py:258-259); see OSV1-005. - id: OSV1-021 - title: Conformance 2 (alarm unmissable, never colour-only) has no test file + title: >- + Conformance 2 (alarm unmissable, never colour-only) -- Tier-A half lands, + Tier-B hue half still has no test file contract: file: contracts/operator-surface.v1.md clause: Conformance 2 @@ -1801,21 +1831,25 @@ kind: probe ref: test_row_osv1_021 notes: > - PINNING ROW -- the probe pins both named paths' ABSENCE; a passing probe - here is NOT conformance. Flip direction VIOLATION-MOVEMENT. + PINNING ROW -- one half has landed, the other has not, so the probe now + pins the TIER-B half's absence rather than both. A passing probe here is + NOT conformance. Flip direction VIOLATION-MOVEMENT. THE ONLY CONFORMANCE FIXTURE THAT SPANS BOTH TIERS, so it is owned by two - items: the Tier-A half (accessible name on every status-bearing element) - by work_item_pipeline-c1a, carried in this row's `work` ref; the Tier-B - half (the alarm region's hue is `--alarm` or `--blocked`) by - work_item_pipeline-qgo. Both must land before this row can go green; the - single-ref schema records the first, and this note records the second so - the second is not lost. - - MEASURED 2026-09-04 against 4aaee50: neither path exists. GOOD = the alarm - region is present, its hue is `--alarm` or `--blocked`, and every - status-bearing element also carries a word. BAD = a fixture whose status - chips carry only a status class fails the accessible-name assertion. + items. Status 2026-09-05: + TIER-A HALF, LANDED (work_item_pipeline-c1a). "every status-bearing + element also carries a word" is implemented as `state.not_colour_only` + in `tests/conformance/operator_surface/test_tier_a.py`, run over + rendered L0/L1/L2 of the ALARM fixture, and it PASSES -- see OSV1-004, + which carries the measurement. Its bad half is this fixture's own named + specimen ("a fixture whose status chips carry only a status class") and + the same function reports it. + TIER-B HALF, NOT BUILT (work_item_pipeline-qgo). "the alarm region is + present, its hue is `--alarm` or `--blocked`" needs a real browser; + `tests/conformance/operator_surface/browser/test_tier_b.py` does not + exist. + Both must land before this row can go green; the single-ref schema records + the first, and this note records the second so the second is not lost. - id: OSV1-022 title: Conformance 3 (body-swap survival) has no test file @@ -1874,7 +1908,9 @@ fixture has a real chance of failing on first run -- which is the point. - id: OSV1-024 - title: Conformance 5 (hero composition) has no test file + title: >- + Conformance 5 (hero composition) -- fixture exists and discriminates, good + half deferred against OSV1-001 contract: file: contracts/operator-surface.v1.md clause: Conformance 5 @@ -1886,23 +1922,35 @@ kind: probe ref: test_row_osv1_024 notes: > - PINNING ROW -- the probe pins the named path's ABSENCE; a passing probe - here is NOT conformance. Flip direction VIOLATION-MOVEMENT. - - MEASURED 2026-09-04 against 4aaee50: `tests/conformance/` does not exist. - GOOD = the hero region contains a velocity figure with its window stated - and each of in flight, blocked, needs attention, and open/ready. BAD = a - hero carrying only a verdict line, or a figure without the four counts, - fails the assertion naming what is missing. + PINNING ROW -- the fixture exists and discriminates, but its GOOD half does + not pass, so the probe pins that deferral. A passing probe here is NOT + conformance. Flip direction VIOLATION-MOVEMENT. - THE BAD HALF IS THE SHIPPED HERO. Measured in OSV1-001: webapp.py:4602 - renders exactly "a hero carrying only a verdict line". So this fixture's - bad half needs no synthetic construction -- it can be built from - `render_verdict_hero`'s pre-fix output, which is the strongest possible - demonstration that the check discriminates. + MEASURED 2026-09-05. `tests/conformance/operator_surface/test_tier_a.py` + implements `hero.velocity_and_counts` at the location Conformance 5 names, + with three bad halves and one good half: + BAD, DEMONSTRATED "a hero carrying only a verdict line" -- and it needed + no synthetic construction, exactly as this row predicted at seed: the + bad fixture IS `widgets.render_verdict_hero`'s own output shape. Also + "a figure without the four counts" (reports all four missing, and does + NOT report the figure missing), and a third that keeps the check from + crediting a loose number and a status word in different sentences as a + presented count. + GOOD, DEFERRED run against L0 rendered from a populated fixture, it + FAILS: the hero states no `blocked` count and no `needs attention` + count. Those live in the KPI strip OUTSIDE the hero. So the good half + is `xfail(strict=True)` naming OSV1-001. + + THIS ROW GOES GREEN WHEN OSV1-001 DOES, and not before: the fixture is the + fixture, but Conformance 5 is only demonstrated end-to-end once its good + half passes against a conforming hero. The probe therefore asserts both + facts -- the fixture is present with its pair, and OSV1-001 is still red -- + so flipping either one alone fails loudly. - id: OSV1-025 - title: Conformance 6 (inline-style register) has no test file + title: >- + Conformance 6 (inline-style register) -- fixture exists and discriminates, + good half deferred against OSV1-005 contract: file: contracts/operator-surface.v1.md clause: Conformance 6 @@ -1914,25 +1962,37 @@ kind: probe ref: test_row_osv1_025 notes: > - PINNING ROW -- the probe pins the named path's ABSENCE; a passing probe - here is NOT conformance. Flip direction VIOLATION-MOVEMENT. + PINNING ROW -- the fixture exists and discriminates, but its GOOD half does + not pass, so the probe pins that deferral. A passing probe here is NOT + conformance. Flip direction VIOLATION-MOVEMENT. - MEASURED 2026-09-04 against 4aaee50: `tests/conformance/` does not exist. - GOOD = zero inline `style=` attributes carry a literal colour, font, or - size; every computed-geometry site is on the register. BAD = a file - carrying `style="color:#D9A253"` is reported as a literal-colour violation, - and a computed-geometry site absent from the register is reported as - unregistered. - - NOTE THE OVERLAP, AND KEEP IT DELIBERATE: this ledger already runs that - census in-process (OSV1-005 / OSV1-006) because the register lives in - `ledger/` by ruling. The Tier-A kit's job is not to duplicate it but to - read the SAME register, so there is one census and one source of the - number. If the kit grows its own copy, the two will disagree and the - disagreement will be silent. + MEASURED 2026-09-05. `tests/conformance/operator_surface/test_tier_a.py` + implements `visual.single_source` at the location Conformance 6 names: + BAD, DEMONSTRATED the contract's own named specimen, `style="color:#D9A253"`, + is classified LITERAL by the census; and an injected computed-geometry + site is reported as unregistered -- injected over the census's OWN + reader, so the real engine does the classifying and nothing on disk is + touched. + GOOD, DEFERRED 66 literal inline sites and 40 `", "", html, flags=re.DOTALL) + html = re.sub(r"]*>.*?", "", html, flags=re.DOTALL) + builder = _TreeBuilder() + builder.feed(html) + return builder.root + + +def walk(node: Node): + for child in node.children: + yield child + yield from walk(child) + + +def find_one(root: Node, *, node_id: str) -> Node | None: + for n in walk(root): + if n.attrs.get("id") == node_id: + return n + return None + + +# =========================================================================== +# Core 1 -- hero.velocity_and_counts +# +# "the rendered L0 hero region contains a velocity figure with its window +# stated, and each of the four named counts." +# +# The four the clause names: in flight (held), blocked, needs attention, and +# open/ready. +# =========================================================================== + +#: Each count, with every word the surface could legitimately label it with. +#: Named in the clause's own order. +_HERO_COUNTS: tuple[tuple[str, tuple[str, ...]], ...] = ( + ("in flight (held)", ("in flight", "in-flight", "held", "holding")), + ("blocked", ("blocked",)), + ("needs attention", ("needs attention", "needs you", "attention", "needs-you")), + ("open/ready", ("open", "ready", "unclaimed")), +) + +#: A throughput word -- the figure itself must be one of these, not any number. +_THROUGHPUT_WORDS = ("resolved", "velocity", "throughput", "completed", "closed", "burn") + +#: A stated window. `24h`/`7d`/`30d`, "last 7 days", "per day", "/day". +_WINDOW = re.compile( + r"(\b\d+\s*(h|hr|hrs|hours?|d|days?|w|weeks?)\b|\bper day\b|/day\b|\blast \d+\b|\btoday\b)", + re.IGNORECASE, +) +_HAS_DIGIT = re.compile(r"\d") + + +def _leaf_phrases(region: Node, *, max_len: int = 140) -> list[str]: + """Short text runs in the region -- a count and its label read together. + + Bounded, because "the hero contains the word `blocked` somewhere and the + digit `3` somewhere" is not a count: the clause asks for the counts to be + PRESENTED, so the number and its label must sit in one readable run. + """ + out: list[str] = [] + for n in walk(region): + txt = " ".join(n.text().split()) + if txt and len(txt) <= max_len: + out.append(txt) + return out + + +def check_hero_velocity_and_counts(l0_html: str) -> list[str]: + """Core 1's `hero.velocity_and_counts`, against a rendered L0.""" + root = parse_html(l0_html) + hero = find_one(root, node_id="verdict-hero") + if hero is None: + return ["no L0 hero region (`#verdict-hero`) in the rendered page at all"] + + phrases = _leaf_phrases(hero) + problems: list[str] = [] + + velocity = [ + p + for p in phrases + if _HAS_DIGIT.search(p) + and any(w in p.lower() for w in _THROUGHPUT_WORDS) + and _WINDOW.search(p) + ] + if not velocity: + problems.append( + "no velocity figure with its window stated -- the hero carries no run " + f"pairing a number, a throughput word {_THROUGHPUT_WORDS} and a window" + ) + + for label, words in _HERO_COUNTS: + if not any(_HAS_DIGIT.search(p) and any(w in p.lower() for w in words) for p in phrases): + problems.append(f"the hero states no `{label}` count") + return problems + + +def test_hero_velocity_and_counts_bad_half_verdict_line_only() -> None: + """Conformance 5's bad half, verbatim: "a hero carrying only a verdict line". + + Not synthesised from nothing -- this IS `widgets.render_verdict_hero`'s + own output shape (OSV1-024's notes: "the bad half needs no synthetic + construction"), rendered around the calm verdict the shipped L0 emits. + """ + bad = """
+
Environment verdict · 21 projects
+
All clear
+
Nothing stuck, nothing waiting past its TTL.
+
""" + problems = check_hero_velocity_and_counts(bad) + assert problems, "the check passed a hero carrying only a verdict line" + assert any("velocity figure" in p for p in problems) + for label, _ in _HERO_COUNTS: + assert any(label in p for p in problems), f"the check did not name `{label}` as missing" + + +def test_hero_velocity_and_counts_bad_half_figure_without_the_counts() -> None: + """Conformance 5's other bad half: "a figure without the four counts".""" + bad = """
+
14 resolved in the last 24h
+
""" + problems = check_hero_velocity_and_counts(bad) + assert not any("velocity figure" in p for p in problems), ( + "the figure IS present with its window -- the check must not report it missing" + ) + assert len(problems) == len(_HERO_COUNTS), ( + f"expected all four counts reported missing, got {problems}" + ) + + +def test_hero_velocity_and_counts_bad_half_a_loose_number_is_not_a_count() -> None: + """A number somewhere and a word somewhere else is not a presented count.""" + bad = """
+
All clear
+
7 resolved in the last 24h.
+
Nothing is blocked, and nothing needs attention, and + the ready queue and everything in flight are described here at length + in a sentence that never actually puts a number beside any of them.
+
""" + problems = check_hero_velocity_and_counts(bad) + assert any("blocked" in p for p in problems), ( + "a status word in a long prose run with no adjacent number was credited as a count" + ) + + +@pytest.mark.xfail( + strict=True, + reason="OSV1-001 (Core 1): the L0 hero is a verdict line; velocity is a chart " + "below it and `blocked`/`needs attention` are only in the KPI strip outside " + "the hero. Flip the row and delete this marker in the same change.", +) +def test_hero_velocity_and_counts(calm_dataset) -> None: + """Conformance 5's GOOD half: L0 rendered against a populated fixture.""" + problems = check_hero_velocity_and_counts(calm_dataset.l0) + assert not problems, "Core 1 (`hero.velocity_and_counts`):\n " + "\n ".join(problems) + + +# =========================================================================== +# Core 3 -- state.not_colour_only +# +# "every status-bearing element in the rendered L0/L1/L2 fixtures has +# non-empty text or an accessible name, not merely a status class." +# =========================================================================== + +#: The status vocabulary, read out of the app's OWN maps rather than written +#: down here: `webbrowse._ITEM_STATUS_CHIP_LABEL`'s keys (the item statuses), +#: `widgets._HERO_ICON`'s keys (the verdict states) and +#: `webapp._ATTENTION_SEVERITY`'s severities. A status the app grows and this +#: kit does not know about would otherwise be silently unchecked. +_STATUS_CLASS_PREFIXES = ("st", "is", "sev", "tab", "status") + + +def _dict_keys_in(path: Path, name: str) -> set[str]: + """The string keys of a module-level `NAME: ... = {...}` dict literal.""" + tree = ast.parse(S.read(path)) + for node in ast.walk(tree): + targets: list[ast.expr] = [] + if isinstance(node, ast.AnnAssign): + targets = [node.target] + elif isinstance(node, ast.Assign): + targets = list(node.targets) + if not any(isinstance(t, ast.Name) and t.id == name for t in targets): + continue + value = node.value if isinstance(node, ast.AnnAssign | ast.Assign) else None + if isinstance(value, ast.Dict): + return { + k.value + for k in value.keys + if isinstance(k, ast.Constant) and isinstance(k.value, str) + } + raise AssertionError(f"{path.name}: no module-level dict literal named {name!r}") + + +def _dict_values_in(path: Path, name: str) -> set[str]: + tree = ast.parse(S.read(path)) + for node in ast.walk(tree): + targets = ( + [node.target] + if isinstance(node, ast.AnnAssign) + else list(node.targets) + if isinstance(node, ast.Assign) + else [] + ) + if not any(isinstance(t, ast.Name) and t.id == name for t in targets): + continue + value = node.value if isinstance(node, ast.AnnAssign | ast.Assign) else None + if isinstance(value, ast.Dict): + return { + v.value + for v in value.values + if isinstance(v, ast.Constant) and isinstance(v.value, str) + } + raise AssertionError(f"{path.name}: no module-level dict literal named {name!r}") + + +def status_vocabulary() -> set[str]: + """Every word this surface uses to name a state, from its own source.""" + words = set(_dict_keys_in(S.WEBBROWSE, "_ITEM_STATUS_CHIP_LABEL")) + words |= {v.lower() for v in _dict_values_in(S.WEBBROWSE, "_ITEM_STATUS_CHIP_LABEL")} + words |= set(_dict_keys_in(S.WIDGETS, "_HERO_ICON")) + # The status hues themselves -- `--alarm` -> `alarm` -- so `is-alarm` and + # `sev-blocked` are recognised even though no dict literal names them. + words |= {t.lstrip("-") for t in S.STATUS_TOKENS} + return {w for w in words if w} + + +def status_classes(rendered: set[str]) -> set[str]: + """Which of a page's classes are STATUS classes. + + A class is status-bearing when it is a status word, or a status word under + one of this surface's status-class prefixes (`st-blocked`, `is-alarm`, + `sev-cr`, `tab-blocked`). The chip vocabulary is taken from + `_ITEM_STATUS_CHIP_CLASS` directly as well, so a chip class that does not + follow the prefix convention is still caught. + """ + vocab = status_vocabulary() + chips = _dict_values_in(S.WEBBROWSE, "_ITEM_STATUS_CHIP_CLASS") + out = {c for c in rendered if c in chips} + for c in rendered: + if c in vocab: + out.add(c) + continue + head, _, tail = c.partition("-") + if head in _STATUS_CLASS_PREFIXES and tail in vocab: + out.add(c) + return out + + +def check_state_not_colour_only(html: str, *, page: str) -> list[str]: + """Core 3's `state.not_colour_only`, against one rendered page.""" + root = parse_html(html) + nodes = list(walk(root)) + rendered_classes = {c for n in nodes for c in n.classes} + status = status_classes(rendered_classes) + if not status: + return [f"{page}: no status-bearing element on the page at all -- nothing was checked"] + + problems: list[str] = [] + for n in nodes: + if not (n.classes & status): + continue + if n.accessible_name(): + continue + # A decorative mark INSIDE a status-bearing element that is itself + # named carries no state of its own -- the state is already in that + # ancestor's words. Colour is redundant there, which is the clause. + if any(a.classes & status and a.accessible_name() for a in n.ancestors()): + continue + problems.append( + f"{page}: {n.describe()} carries status class(es) " + f"{sorted(n.classes & status)} and NO text or accessible name" + ) + return problems + + +def test_state_not_colour_only_bad_half_chip_with_only_a_class() -> None: + """Conformance 2's bad half, verbatim: "a fixture whose status chips carry + only a status class fails the accessible-name assertion".""" + bad = '
' + problems = check_state_not_colour_only(bad, page="BAD") + assert problems, "a status chip carrying only a class was accepted" + assert "st-blocked" in problems[0] + + +def test_state_not_colour_only_bad_half_an_accessible_name_is_enough() -> None: + """The other direction: a chip with no text but a real name PASSES. + + Without this the check would be asserting "has text", not the clause's + "non-empty text OR an accessible name" -- and would fail the surface for + an icon-only control that is correctly labelled. + """ + good = '' + assert not check_state_not_colour_only(good, page="GOOD") + + +def test_state_not_colour_only_bad_half_an_ancestors_name_does_not_travel_sideways() -> None: + """A named status ancestor excuses only its own decorative descendants.""" + bad = ( + # A decorative, unlabelled mark INSIDE a named status control: excused. + 'Blocked 3' + # A status chip standing on its own with nothing at all: reported. + '' + ) + problems = check_state_not_colour_only(bad, page="BAD") + assert len(problems) == 1 and "st-held" in problems[0], problems + + +@pytest.mark.parametrize("level", ["l0", "l1", "l2"]) +def test_state_not_colour_only(alarm_dataset, level: str) -> None: + """Conformance 2's Tier-A GOOD half, on all three IA levels. + + Run against the ALARM dataset, which is Conformance 2's own scenario -- + "the same fixture with one item held past TTL and one blocked" -- so every + status chip the surface can render is actually on the page. + """ + if level == "l2": + html = alarm_dataset.l2(alarm_dataset.item_ids[1]) # the blocked item + else: + html = getattr(alarm_dataset, level) + problems = check_state_not_colour_only(html, page=f"{alarm_dataset.label} {level.upper()}") + assert not problems, "Core 3 (`state.not_colour_only`):\n " + "\n ".join(problems) + + +# =========================================================================== +# Core 4 -- visual.single_source +# +# "zero inline `style=` attributes carrying a literal colour, font, or size +# anywhere in `src/`; every inline `style=` site carrying computed geometry +# appears on the ledger's exemption register; and zero literal colour/font/ +# size declarations in any `