Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,69 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this project a

## [Unreleased]

### Fixed, five fields published a zero nobody had ruled on, and the registry could not say so

- **A reviewed decision and an unexamined field published the same thing.** `NOOFCARSONPROPERTY`,
`NOOUTBUILDINGSDAMAGED`, `NOOUTBUILDINGSNOTDAMAGED`, `NUMBEROFUNITPERSTRUCTURE` and
`ASSESSEDIMPROVEDVALUE` are each `FieldSpec(..., numeric=True)` with no declared
vocabulary, so each carried `Basis.NONE` and published `marker_basis: "none"`. So does a
field nobody has ever opened. `docs/MARKERS.md` section 7 named the problem in its own
last sentence: *"the gate that reads this file has no way to tell a reviewed decision
from an unexamined one."* Four of the five record a zero in over 94% of the cells they
record anything in, and on the measurement reading `NOOFCARSONPROPERTY` says 357 of
56,188 inspections found a car.

- **`FieldSpec.zero_reading` is the second axis.** `ZeroReading` has four members and the
default is `UNREVIEWED`: `measurement` (a zero is a finding), `undecidable` (reviewed,
and the published documentation does not settle it, so the zero is left present and must
not be read either way), `marker` (declared, so the zeros are counted as
recorded-as-unknown and the field publishes none), and `unreviewed` (nobody has ruled).
It is a second axis rather than more `Basis` members because `Basis` answers *where a
declared vocabulary came from* and this answers *what a zero is*; a field can be in any
combination of the two, and folding them is the same conflation one column over.

- **The readings are the ones section 7 already recorded**, transcribed into the registry
rather than invented: three counts are `measurement`, the two contested fields are
`undecidable`, and `YEARBUILT` is `marker`. Section 7 now states the word beside each
row, and `tests/test_schema.py` holds the document and the registry to each other in
both directions by an exact token.

- **Two numbers on every surface, derived from the rows they describe.** Each artifact
carries a `recorded_zero_review` block: fields measured as numbers, how many carry a
reviewed reading, which ones do not (named, not counted away), and the same pair
restricted to the fields actually publishing a zero. `dins.html` and `perimeters.html`
state it above each field table, and every field's row carries its own reading beside
its zero count. Measured on this retrieval: **6 of 8** DINS fields measured as numbers
carry a reading and **5 of 5** of those publishing a zero do; on the FRAP side **0 of
1**, and `GIS_ACRES` publishes no zero. `LATITUDE` and `LONGITUDE` are the DINS pair
with no reading, and they publish none either.

- **The gate.** A field publishing a recorded zero while declaring `unreviewed` fails the
build, with a message naming the two files to change. The older gate asked only whether
a field was *named* somewhere in `docs/MARKERS.md`, which a field can be while the
document says nothing about its zeros. Both are kept: they catch different things, and
the new one is exercised by a positive control built from the five fields exactly as
they stood when issue #83 was filed.

- **Two gates that would have passed over the change being made.** Both were found by
sabotaging the tree and watching nothing go red. Withdrawing `NOOFCARSONPROPERTY`'s
ruling in `src/perimeter/schema.py`, so the field publishes 55,831 zeros while declaring
that nobody ruled on them, left all 534 tests in `tests/test_schema.py` green: every gate
there reads the *committed* artifact, and the only thing that rebuilds it — `make
site-check` — needs `data/raw/` and cannot run in CI. The published artifact's spine
comparison, which already holds `marker_basis` between a fixture build and the committed
copy, now holds `recorded_zero_reading` beside it. And counting every examinable field as
reviewed, so the block publishes `8 of 8`, left all 1,158 tests green, because every gate
on that block recomputes it from the same payload it was written into;
`zero_review` now has two tests that read its own answer over a field set whose two
numbers are known to differ.

- **`artifact_schema_version` is now 2.** `recorded_zero_review` is a new required
top-level key and `recorded_zero_reading` a new optional per-field one, so a consumer
validating against version 1 — whose schema closes with `additionalProperties: false` —
would reject the new artifact. That is the stated rule for a bump, and it is stated here
and in `src/perimeter/artifacts.py` so neither can be edited alone.

### Fixed, the secret scan read one of `main`'s eighty-one commits and reported success

- **`secret-scan` was `gitleaks/gitleaks-action`, which picks its scan range from the
Expand Down
46 changes: 39 additions & 7 deletions docs/MARKERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -275,11 +275,28 @@ as well: the count in the sentence above, the set of rows, and every number in t

| Field | Recorded zeros | Reading | Declared? |
|---|---|---|---|
| `NOOUTBUILDINGSDAMAGED` | 59,628 | a count of no outbuildings damaged | no, a zero is the measurement |
| `NOOUTBUILDINGSNOTDAMAGED` | 58,888 | a count of no undamaged outbuildings | no, a zero is the measurement |
| `NOOFCARSONPROPERTY` | 55,831 | a count of no cars | no, a zero is the measurement |
| `NUMBEROFUNITPERSTRUCTURE` | 58,411 | contested; see below | no, and the reason is recorded |
| `ASSESSEDIMPROVEDVALUE` | 6,613 | contested; see below | no, and the reason is recorded |
| `NOOUTBUILDINGSDAMAGED` | 59,628 | a count of no outbuildings damaged | `measurement` : no marker declared; a zero is the finding |
| `NOOUTBUILDINGSNOTDAMAGED` | 58,888 | a count of no undamaged outbuildings | `measurement` : no marker declared; a zero is the finding |
| `NOOFCARSONPROPERTY` | 55,831 | a count of no cars | `measurement` : no marker declared; a zero is the finding |
| `NUMBEROFUNITPERSTRUCTURE` | 58,411 | contested; see below | `undecidable` : reviewed, no marker declared, and the reason is recorded |
| `ASSESSEDIMPROVEDVALUE` | 6,613 | contested; see below | `undecidable` : reviewed, no marker declared, and the reason is recorded |

The word in the last column is the one the registry carries in
`FieldSpec.zero_reading` and the artifacts publish as `recorded_zero_reading`, and
`tests/test_schema.py` holds the two to each other in both directions. Before it existed
the registry had no way to say any of this: a field whose zeros a reviewer had read the
form for and left as measurements, and a field nobody had ever opened, both carried
`Basis.NONE` and published `marker_basis: "none"`. That is the last sentence of this
section, and issue #83 is it being fixed.

**What a reader should conclude from each word.**

| Word | What it says | What a reader should do with a `0` |
|---|---|---|
| `measurement` | Somebody read the inspection form and the zeros' distribution, and a recorded `0` is a finding | Use it as a count of none |
| `undecidable` | Somebody read them and the published documentation does not settle it. Both readings are live and the file cannot separate them, so the zero is **left present** : declaring it a marker would move tens of thousands of records on a reading nothing supports | Do not use it as a finding, and do not use it as an absence. `recorded_zero_values` is published so either reading can be applied |
| `marker` | Declared: `0` stands in for a value that was not recorded, so the zeros are counted as recorded-as-unknown and this field publishes none | There is no `0` to read; it is already counted as an absence |
| `unreviewed` | **Nobody has ruled.** A recorded `0` would be published as a value and might be an absence | Nothing : and nothing can be, which is why the build fails if a field in this state ever publishes one |

`YEARBUILT` was the sixth row of that table, and it is the reason the checking above
exists. Declaring `0` a marker for it moved all 12,148 of its zeros out of
Expand Down Expand Up @@ -391,5 +408,20 @@ from scratch.
and a count of zero is what an inspector writes when there were none. Their positive
values run from 1 upward in the shape a real count has. Nothing is declared and nothing
moves; they are listed here because "we looked and a zero is a measurement" is a finding a
reader is entitled to, and because the gate that reads this file has no way to tell a
reviewed decision from an unexamined one.
reader is entitled to.

`NOOFCARSONPROPERTY` is the hardest of the three and is recorded as such: 55,831 zeros
against 357 positives is 99.4% of everything the field records, which on the measurement
reading means 357 of the 56,188 inspections that wrote anything at all found a car. It is
left as a measurement because that is what the distribution of the positives supports and
because no published documentation offers a placeholder convention to read instead : not
because the share is comfortable. **This is the second entry to revisit**, after
`ASSESSEDIMPROVEDVALUE`, and the reading is now carried in the registry rather than only
here, so changing it is a one-line change with a gate on both sides of it.

The last sentence of this section used to read *"and because the gate that reads this file
has no way to tell a reviewed decision from an unexamined one."* It now has one:
`FieldSpec.zero_reading`, published per field as `recorded_zero_reading`, summarised in
each artifact's `recorded_zero_review` block as two numbers, and stated on the page above
each field table. A field measured as a number that publishes a recorded zero while
carrying `unreviewed` fails the build.
2 changes: 1 addition & 1 deletion site/data/datapackage.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"artifactSchemaVersion": 1,
"artifactSchemaVersion": 2,
"description": "Counts of cells in three named states across CAL FIRE's historical fire perimeter record and its damage inspection record. Unofficial. Both publishers state in their own words that these files are incomplete; this measures how, in counts, with no estimation and no share over an empty denominator.",
"fieldStateMeaning": "The three counts are in the order published as `field_state_order`: present is a recorded value, explicit_unknown is a value the publisher records to mean it could not be determined, and not_recorded is an empty cell. A recorded unknown is neither present nor missing, and folding it into either one is the error this project exists to refuse.",
"fieldStateOrder": [
Expand Down
20 changes: 19 additions & 1 deletion site/data/dins-coverage.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"inaccessible": 591,
"total": 132522
},
"artifact_schema_version": 1,
"artifact_schema_version": 2,
"completeness_by_access": [
{
"assessed_present": 131931,
Expand Down Expand Up @@ -53497,6 +53497,7 @@
"outside_published_domain_values_listed": null,
"present": 61929,
"present_tenths_pct": 467,
"recorded_zero_reading": "undecidable",
"recorded_zero_values": 58411,
"total": 132522
},
Expand All @@ -53513,6 +53514,7 @@
"outside_published_domain_values_listed": null,
"present": 61939,
"present_tenths_pct": 467,
"recorded_zero_reading": "measurement",
"recorded_zero_values": 59628,
"total": 132522
},
Expand All @@ -53529,6 +53531,7 @@
"outside_published_domain_values_listed": null,
"present": 61958,
"present_tenths_pct": 468,
"recorded_zero_reading": "measurement",
"recorded_zero_values": 58888,
"total": 132522
},
Expand All @@ -53545,6 +53548,7 @@
"outside_published_domain_values_listed": null,
"present": 56188,
"present_tenths_pct": 424,
"recorded_zero_reading": "measurement",
"recorded_zero_values": 55831,
"total": 132522
},
Expand Down Expand Up @@ -53604,6 +53608,7 @@
"outside_published_domain_values_listed": null,
"present": 89943,
"present_tenths_pct": 679,
"recorded_zero_reading": "marker",
"recorded_zero_values": 0,
"total": 132522
},
Expand All @@ -53620,6 +53625,7 @@
"outside_published_domain_values_listed": null,
"present": 126482,
"present_tenths_pct": 954,
"recorded_zero_reading": "undecidable",
"recorded_zero_values": 6613,
"total": 132522
},
Expand Down Expand Up @@ -53686,6 +53692,7 @@
"outside_published_domain_values_listed": null,
"present": 132522,
"present_tenths_pct": 1000,
"recorded_zero_reading": "unreviewed",
"recorded_zero_values": 0,
"total": 132522
},
Expand All @@ -53702,6 +53709,7 @@
"outside_published_domain_values_listed": null,
"present": 132522,
"present_tenths_pct": 1000,
"recorded_zero_reading": "unreviewed",
"recorded_zero_values": 0,
"total": 132522
}
Expand Down Expand Up @@ -146015,6 +146023,16 @@
"field": "UTILITYMISCSTRUCTUREDISTANCE",
"note": "How often each declared spelling of this field's Not Applicable value was written, per incident-start year. Both spellings are counted as the publisher's finding rather than one being read as missing data; docs/MARKERS.md section 2 sets out the evidence and the confidence."
},
"recorded_zero_review": {
"fields_measured_as_numbers": 8,
"fields_publishing_a_recorded_zero": 5,
"fields_publishing_a_recorded_zero_with_a_reviewed_reading": 5,
"fields_with_a_reviewed_zero_reading": 6,
"fields_without_one": [
"LATITUDE",
"LONGITUDE"
]
},
"records": 132522,
"records_not_attributable_to_an_incident": 0,
"source": {
Expand Down
12 changes: 11 additions & 1 deletion site/data/perimeters-coverage.json
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@
300
]
},
"artifact_schema_version": 1,
"artifact_schema_version": 2,
"duplicate_signals": [
{
"description": "Records sharing one IRWIN ID. IRWIN is the federal incident identifier, so a repeat means two perimeter records point at one federal incident.",
Expand Down Expand Up @@ -427,6 +427,7 @@
"outside_published_domain_values_listed": null,
"present": 23334,
"present_tenths_pct": 1000,
"recorded_zero_reading": "unreviewed",
"recorded_zero_values": 0,
"total": 23334
},
Expand Down Expand Up @@ -494,6 +495,15 @@
}
],
"measurement": "Completeness of the FRAP historical fire perimeter record",
"recorded_zero_review": {
"fields_measured_as_numbers": 1,
"fields_publishing_a_recorded_zero": 0,
"fields_publishing_a_recorded_zero_with_a_reviewed_reading": 0,
"fields_with_a_reviewed_zero_reading": 0,
"fields_without_one": [
"GIS_ACRES"
]
},
"records": 23334,
"records_without_year": 77,
"source": {
Expand Down
50 changes: 49 additions & 1 deletion site/data/schema/dins-coverage.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
"type": "object"
},
"artifact_schema_version": {
"const": 1,
"const": 2,
"description": "The version of this contract. Bumped when a consumer validating against the previous schema would reject the new artifact or would read an existing key as meaning something else.",
"type": "integer"
},
Expand Down Expand Up @@ -597,6 +597,16 @@
"null"
]
},
"recorded_zero_reading": {
"description": "What a recorded zero in this field MEANS, as a reviewer has ruled. `measurement`: a zero is a finding. `undecidable`: reviewed, and the published documentation does not settle it, so the zero is left present and must not be read as a finding OR as an absence. `marker`: declared as a marker, so this field's zeros are counted in `explicit_unknown` and `recorded_zero_values` is 0. `unreviewed`: NOBODY HAS RULED -- a zero here may be an absence, and the build fails if such a field ever publishes one. Read beside `recorded_zero_values`, never instead of it, and see docs/MARKERS.md section 7. Absent for a field that is not numeric.",
"enum": [
"unreviewed",
"measurement",
"undecidable",
"marker"
],
"type": "string"
},
"recorded_zero_values": {
"description": "Cells recording a literal zero, for a field measured as a number. A zero is a judgment call (ADR 0006) and is published separately rather than folded into either presence or absence. Absent for a field that is not numeric.",
"type": "integer"
Expand Down Expand Up @@ -794,6 +804,43 @@
],
"type": "object"
},
"recorded_zero_review": {
"additionalProperties": false,
"description": "Coverage of the zero question over this artifact's fields. Every figure is derived from the fields beside it; none is stated anywhere.",
"properties": {
"fields_measured_as_numbers": {
"description": "The denominator: every field that can carry a zero, and so every field this project either has or has not ruled on.",
"type": "integer"
},
"fields_publishing_a_recorded_zero": {
"description": "Of the fields measured as numbers, how many actually carry a zero in this retrieval.",
"type": "integer"
},
"fields_publishing_a_recorded_zero_with_a_reviewed_reading": {
"description": "Of those, how many carry a reading. Smaller than the figure above means a zero nobody has ruled on is being published as a value, which is the condition `tests/test_schema.py` fails the build on.",
"type": "integer"
},
"fields_with_a_reviewed_zero_reading": {
"description": "Of those, how many carry a `recorded_zero_reading` other than `unreviewed`. `undecidable` counts as reviewed: a review that reached no verdict is still a review, and it is a finding rather than a gap.",
"type": "integer"
},
"fields_without_one": {
"description": "The rest, named rather than counted away. A zero appearing in any of these fails the build until somebody rules on it.",
"items": {
"type": "string"
},
"type": "array"
}
},
"required": [
"fields_measured_as_numbers",
"fields_publishing_a_recorded_zero",
"fields_publishing_a_recorded_zero_with_a_reviewed_reading",
"fields_with_a_reviewed_zero_reading",
"fields_without_one"
],
"type": "object"
},
"records": {
"description": "Records measured.",
"type": "integer"
Expand Down Expand Up @@ -957,6 +1004,7 @@
"is_fixture",
"measurement",
"not_applicable_spellings_by_year",
"recorded_zero_review",
"records",
"records_not_attributable_to_an_incident",
"source"
Expand Down
Loading
Loading