Skip to content

The artifacts are the product and had no contract but this repository's source - #79

Merged
ChelseaKR merged 1 commit into
mainfrom
feat/artifact-schemas-and-datapackage
Sep 7, 2026
Merged

ChelseaKR merged 1 commit into
mainfrom
feat/artifact-schemas-and-datapackage

Conversation

@ChelseaKR

Copy link
Copy Markdown
Owner

What was wrong

site/data/*.json are the product. The pages are one rendering of them, and the sibling project, a researcher or a reporter had no contract for their shape except this repository's source.

The sharpest version of that: the three-state model is the entire point of this measurement, and it is invisible in the JSON.

"EAVES": [222, 0, 0]

That is unreadable without perimeter.artifacts.FIELD_STATE_ORDER, and nothing in the artifact said so. A consumer who guessed the order wrong would fold a recorded unknown into either presence or absence — the exact error this project exists to refuse — and nothing would tell them.

What changed

site/data/schema/perimeters-coverage.schema.json and dins-coverage.schema.json — JSON Schema (draft 2020-12), written by the same run that writes the artifacts, so a schema can never describe a build that produced something else. They name every key, say which of the three counts is a recorded value, which is a published marker meaning the value could not be determined and which is an empty cell, and say why a share is null rather than 0 over an empty denominator.

site/data/datapackage.json — a Frictionless Data Package naming both resources with each source's licence, endpoint, layer, retrieval date, byte count, record count and SHA-256. Those come from perimeter.sources, the single reviewed provenance record, so a descriptor cannot state a hash the artifacts do not — held there the way tests/test_provenance.py holds PROVENANCE.md. A fixture build writes isFixture: true with every acquisition fact null: a fixture was never downloaded from anywhere, and a descriptor is the most quotable place a fixture could pass itself off as a measurement of CAL FIRE's files, because it is the file a machine reads rather than a person.

artifact_schema_version in both artifacts, with the bump rule stated beside the constant and in CHANGELOG.md, and a test holding the two together so neither can be edited alone.

The footer of all three pages links the descriptor and both schemas.

Every existing number is unchanged

Rebuilt with make site against the acquired FRAP and DINS files (both present locally, both SHA-256-matching sources.py). make diff against origin/main:

perimeters-coverage.json   added: 1   /artifact_schema_version: not published -> 1
                           1175 leaves before, 1176 after
dins-coverage.json         added: 1   /artifact_schema_version: not published -> 1
                           56241 leaves before, 56242 after

One leaf each, and nine footer lines per page. That is also independent confirmation that the committed site/ was byte-current with origin/main before this branch touched it.

A deviation from the issue, stated plainly

#63 asks for schemas "generated from the dataclasses in coverage.py and artifacts.py". They are declared in schema_export.py instead, because the payloads are not dataclasses: perimeters_payload and dins_payload assemble dictionaries by hand, add derived keys (total, present_tenths_pct), rename others (outside_domain becomes outside_published_domain) and drop two conditionally. A generator over FieldCoverage would describe the dataclass and not the artifact, which is the wrong document.

What the issue actually asks for is that the schema "cannot drift from the writer", and that is delivered a different way: validation in both directions against a real build. Every object closes with additionalProperties: false and requires every key it declares, so

  • a key the writer starts emitting fails as an unexpected property, and
  • a key the schema starts demanding fails as a missing one.

That leaves exactly one case uncovered — a key the schema marks optional that nothing emits, which would sit in the contract forever failing nothing. There are two optional keys, both written conditionally by _field_json, and test_every_optional_key_is_one_the_writer_can_omit pins the set to those two by name and proves each is present for some field of the published artifact and absent for another.

How it was verified

Full suite 923 passed. html-validate, tools/a11y.mjs (six rule sets, three pages, nothing undecided beyond the four declared rules) and the Chromium Playwright gate (axe plus reflow at 320px, 8 passed) all green against a fixture build with the new footer.

Per ADR 0004 the contract is run against eight mutations it must refuse, including the case #63 names verbatim: a fixture artifact with field_state_order removed fails, and the error names the path.

Five negative controls, each committed first, baselined with git hash-object, sabotaged, asserted changed, run with __pycache__ cleared under PYTHONDONTWRITEBYTECODE=1, restored and re-hashed:

sabotage expected red result
_object: additionalProperties: FalseTrue closure + both unexpected-key cases 4 red
_resource: the descriptor's real SHA-256 → "sha256:0" the provenance binding 3 red
_resource: None if is_fixture else source.record_count → always the real count the fixture-publishes-no-acquisition-facts guard 1 red
ARTIFACT_SCHEMA_VERSION = 12 version bumped without rebuilding the artifacts 5 red
_field_json emits an undeclared marker_confidence key the fixture build stops validating 3 red

All five restored to their exact baseline blob hashes.

One control was botched on its first attempt and I am reporting it rather than only its second run. The fifth sabotage was applied through a shell argument containing a literal \n, which produced a SyntaxError instead of a valid mutation — the suite went red on a collection error, which would have read as a passing control if I had only looked at "did it go red". It was redone with a heredoc and ast.parse asserting the sabotaged source still parses, so the red came from validation and not from a broken file. Worth recording: "the suite went red" is not the same measurement as "the guard fired".

Also worth noting the fourth control's shape: test_the_committed_artifact_validates went red but test_the_fixture_build_validates stayed green, because the committed bytes and a fresh build disagree only when the writer moves without a rebuild. That is the pair working as intended rather than a flake.

Scope

Part of #63. Not included, and not attempted: publishing either document to a registry or portal, which the issue puts out of scope.

Prepared with AI assistance; reviewed before submission.

…'s source

site/data/*.json are what a downstream reader consumes and the pages are one
rendering of them. Their shape was readable only out of render.py, and the
three-state model -- the whole point of the measurement -- was invisible in the
JSON: [222, 0, 0] means nothing without FIELD_STATE_ORDER, and nothing in the
artifact said so.

Adds JSON Schema (draft 2020-12) per artifact under site/data/schema/, a
Frictionless data package at site/data/datapackage.json carrying each source's
licence, endpoint, retrieval date, byte count, record count and SHA-256 from
the reviewed record in sources.py, and artifact_schema_version in both
artifacts with the bump rule stated beside the constant and in CHANGELOG.md.

The schemas are declared rather than generated, because the payloads are not
dataclasses. What holds them to the writer is validation in both directions
against a real build: every object closes with additionalProperties false and
requires every key it declares. The one case that leaves uncovered, an optional
key nothing emits, is pinned by name to the two keys _field_json writes
conditionally.

Every existing number is unchanged. The rebuild from the acquired files moved
exactly one leaf in each artifact.
@ChelseaKR
ChelseaKR merged commit 71a33ec into main Sep 7, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant