Add an artifact diff command so a deliberate refresh is compared value by value

[ChelseaKR]

What

A refresh of the pinned retrievals replaces site/data/perimeters-coverage.json and site/data/dins-coverage.json wholesale, and today the only thing that says what moved is git diff over a large JSON document. python -m perimeter.diff OLD.json NEW.json walks every leaf of both artifacts, reports each changed value with its path, old value and new value, reports keys added, and refuses to pass when a published key disappeared unless --allow-removals names that as deliberate. Output is console and --json; exit 0 no change, 1 changes reported, 2 removals not allowed or unreadable input. make site-check prints the diff when the served artifact and a fresh build disagree.

Why it matters

The README promises figures "move only when those retrievals are deliberately refreshed", and a deliberate refresh should be able to say what it moved. The sibling wildfire-service-territory-overlap built exactly this (artifact_diff.py) and used it to show a 4,370-value comparison with nothing removed; perimeter is its upstream and should hold itself to the same discipline. The command is offline, deterministic, and reads only the two files given. A field moving from a number to null (a domain the layer stopped publishing, per ADR-0010) is reported as a change to absence, not as a removal.

Scope

• src/perimeter/diff.py and a make diff OLD=... NEW=... target; --allow-removals, --json, --ignore is_fixture for comparing a fixture build to itself.
• Leaf comparison with integer-exact equality; tenths_pct fields compared as integers, never as floats.
• A comparison artifact ({path, old, new} rows sorted by path) that PROVENANCE.md can cite for a refresh.
• Tests over artifacts that must fail it (a removed key, a changed count, a type change, an empty file), per ADR-0004.

Out of scope

• Diffing the pages; they are rendered from the artifacts.
• Automatic refreshes.

Done when

• Diffing an artifact against itself exits 0 and prints "no change".
• Removing duplicate_signals from the new artifact exits 2 without --allow-removals and 1 with it, naming the key.
• A records count change is reported with both values, and the JSON output is byte-identical on repeat.
• make site-check on a deliberately altered site/data/ prints the leaf-level diff.

Pointers

• src/perimeter/artifacts.py, cli.py, Makefile (site-check), PROVENANCE.md
• wildfire-service-territory-overlap: src/wildfire_service_territory_overlap/artifact_diff.py, tests/test_artifact_diff.py

Proposed with AI assistance.

[ChelseaKR]

Triage against origin/main @ d4f533f (2026-09-06). Not on main — no src/perimeter/diff.py. In flight as PR #71, which implements this issue's scope and reports every "Done when" row as checked, including the three distinctions that make the design right (a removed key is refused rather than reported as a change; a number becoming null is a change to absence, not a removal; 1000 and 1000.0 are a type change).

That PR is currently red for a reason unrelated to its own diff: tests/test_release_claims.py refuses to read the tag list because git rev-parse --is-shallow-repository reports true inside the verify job, even though actions/checkout runs with fetch-depth: 0 and the logged fetch carries no --depth. Measured on a runner: the checkout is not shallow, and .git/shallow (containing main's own tip) appears part-way through the test run. Being tracked separately; it is a repository-wide CI defect, not a defect in #71.

Two things in PR #71 worth reading rather than skimming, because they are the kind of thing that usually goes unwritten:

• make site-check prints the leaf comparison and discards its exit status deliberately, so the reporting step can never turn diff -r green. The comparison is the report; the byte-for-byte check is still the gate.
• Its second negative control initially passed for the wrong reason: with the emptiness check deleted, an empty file still failed as unparseable JSON, and pytest.raises(match="empty") was satisfied by the fixture's own filename, empty.json. Renaming the fixture made the control fail as it should. A control that passes for the wrong reason is a control that is not there.

Triaged with AI assistance.