Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,38 @@ jobs:
- name: Tier 5 -- tool module tests
run: .venv/bin/python -m pytest modules/tool-work-tracker/tests -v

# Tier 7 -- operator-surface conformance, Tier B (real browser).
# contracts/operator-surface.v1.md Freeze 2 requires the Tier-B kit to
# "run as its own CI tier", and this is that tier: a separate step, so a
# browser failure is legible at a glance as a RENDERED-SURFACE failure
# and its cost/flake profile never rides inside the fast tiers above.
# Chromium is installed here rather than in the shared setup step for
# the same reason -- nothing above this line needs a browser.
#
# The kit is deselected everywhere else by pyproject's `-m "not tier_b"`
# addopts; `-m tier_b` on this command line is what selects it.
#
# `if: always()` is deliberate: this tier's whole job is to report what
# the rendered surface actually does, and a red tier above (a unit-test
# regression, say) must not hide that report.
- name: Tier 7 -- operator-surface conformance (Tier B, browser)
if: always()
run: |
.venv/bin/playwright install --with-deps chromium
.venv/bin/python -m pytest -m tier_b tests/conformance/operator_surface/browser -v

# The Tier-B artifacts (pixel counts, computed contrast ratios, bounding
# boxes, post-swap DOM snapshots, screenshots) -- Freeze 3's "artifacts
# the orchestrator re-checks itself". Uploaded on every run, pass or
# fail: a failing run's numbers are the ones worth reading.
- name: Tier 7 artifacts -- operator-surface Tier-B evidence
if: always()
uses: actions/upload-artifact@v4
with:
name: operator-surface-tier-b-artifacts
path: tests/conformance/operator_surface/browser/_artifacts/
if-no-files-found: warn

- name: Dolt server log (always, for debugging)
if: always()
run: cat /tmp/dolt-server.log || true
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,8 @@ DONE.json

# goal-batch orchestration scratch (per-run, never repo content)
.amplifier/goals/

# Tier-B conformance artifacts -- a RUN's evidence (pixel counts, contrast
# numbers, post-swap DOM snapshots, screenshots), never repo content. A
# committed artifact is a number nobody re-measured.
tests/conformance/operator_surface/browser/_artifacts/
27 changes: 26 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
.PHONY: venv test test-unit test-integration test-cli test-ledger ledger-mutate test-conformance-a test-module check lint types doctor clean
.PHONY: venv playwright-install test test-unit test-integration test-cli test-ledger \
test-conformance-a test-conformance-b ledger-mutate test-module check lint types \
doctor clean

PYTHON ?= python3.12
VENV := .venv
Expand Down Expand Up @@ -61,6 +63,29 @@ ledger-mutate:
test-conformance-a:
$(PYTEST) tests/conformance/operator_surface -v

## Chromium for the Tier-B browser conformance kit. Separate from `venv`
## on purpose: `playwright` (the library) is a normal dev dependency and
## installs with everything else, but the BROWSER is a ~150MB download into
## `~/.cache/ms-playwright` that a contributor who never runs the browser
## tier should not pay for. `--with-deps` pulls the system libraries chromium
## needs on a bare CI runner (it is a no-op where they are already present).
playwright-install:
$(VENV)/bin/playwright install --with-deps chromium

## Tier B -- operator-surface conformance in a REAL BROWSER
## (contracts/operator-surface.v1.md Freeze 2). Its own target, and
## deliberately NOT part of `test` below: it costs a chromium launch and a
## live app boot per scenario, and Freeze 2 requires it to run as its own
## tier so a browser tier's cost and flake profile never ride inside the fast
## ones. `-m tier_b` overrides the `-m "not tier_b"` in pyproject's addopts
## (pytest inserts addopts before the command line, and the last -m wins), so
## every other invocation deselects this tier automatically.
##
## Depends on `playwright-install`: a Tier-B run without chromium is not a
## pass and not a skip, it is an error naming the missing browser.
test-conformance-b: playwright-install
$(PYTEST) -m tier_b tests/conformance/operator_surface/browser -v

## Tier 5 -- tool module: modules/tool-work-tracker's own suite, the only
## place the post-reclaim custody behaviour of the AGENT SEAM (work_claim /
## work_declare / work_resolve / work_release) is asserted mechanically.
Expand Down
Loading
Loading