Skip to content

feat: a history verb, and the gaps it will not join - #65

Merged
ChelseaKR merged 1 commit into
mainfrom
feat/a-history-verb-from-committed-reports
Sep 6, 2026
Merged

feat: a history verb, and the gaps it will not join#65
ChelseaKR merged 1 commit into
mainfrom
feat/a-history-verb-from-committed-reports

Conversation

@ChelseaKR

Copy link
Copy Markdown
Owner

What was missing

The watch writes one diff per revision under data/changes/ and the reviewed
baseline under data/parsed/. Nothing read them back. The project's thesis is a
cited change feed, and a feed is only useful if a reader can ask "what has this
price been, and when did it move" without re-reading every pull request.

What changed

ca-tariff-parse history --id smud-r-tod \
  --match 'kind=energy_usage label="Generation" season=Summer'

prints every state a matching record has held, each with the retrieval date and
the citation of the revision that set it, ending with the reviewed baseline.
--all covers every record the reports mention; --jsonl writes one object per
timeline.

Everything is built from what is committed. Nothing is fetched, nothing is
interpolated. What it refuses to do is the substance:

A gap is reported, not joined. Each report states the digest of the bytes on
both sides. When one report's "before" digest is not the previous report's
"after" digest, a revision is missing between them — the watch failed to
download it, or the reports were not all committed — and the timeline says so at
that point. The values on either side are real; the line between them is not.
The same check runs against the reviewed baseline, so a baseline written from
bytes no report produced is a gap too.

The order comes from the reports, not the filenames. A retrieval date is a
fact the report states, so it is read from the report. Two reports whose dates
run backwards are refused with both dates named, because the order to read them
in is exactly what is in doubt.

No leg claims one parser read both sides. Each leg carries the three-state
parser_comparison from #59, which has no state meaning "the same parser". The
indeterminate note says in terms that equal release stamps prove nothing.

A record no report mentions is listed with one state, from the baseline,
rather than omitted — otherwise "no result" would mean both "no such record" and
"a record the publisher has not moved".

A --match term naming a field no record kind is identified by is an error,
not a term that quietly matches nothing; a typo would look exactly like a value
the document does not state. A term naming a real field of a different kind
simply does not select that kind. A well-formed match selecting nothing exits
5, a different code from "the reports could not be read".

One supporting change to diff

diff --jsonl now repeats both sides' retrieval date, digest, parser version
and parser_comparison on every change line. It is still one object per
change
— the existing contract and its test are unchanged. The stamps travel
per line rather than in a header so a line lifted out of a report still says
which retrieval it came from, and so history reads a retrieval date rather
than inferring it from a filename, which is the kind of inference this project
does not make.

How it was verified

ruff check, ruff format --check, mypy (strict, 29 files), pytest — all
green, coverage 93.84% against an 85% floor. 26 new tests. No golden file or
baseline changed.

The issue's three "Done when" clauses, exercised over a synthetic scenario of
one baseline and two change reports: a changed price lists three states
(26.20 → 27.10 → 28.40) plus the baseline; an unchanged one lists one;
non-monotonic retrieval dates are refused with both dates named; a --match
that matches nothing exits non-zero and says so.

Negative controls, each read back out of the file before the gate ran, then
restored from a pre-edit byte copy (diff clean) and re-run green:

sabotage gate
chain gaps not detected red — 2 tests
backwards retrieval dates accepted red
an unknown --match field silently matches nothing red
an absent value rendered blank instead of named red (see below)
the parser note claims "the same parser read both sides" red — 2 tests

The fourth row found a real weakness in my own test first, and it is worth
naming.
On its first run the sabotage came back green: the test asserted
that (not in the document) appeared somewhere in the output, and that same
phrase was also produced by a completely different branch — the "now" line for a
record missing from the baseline. So the test passed while a removal rendered as
an empty value. I gave the two branches distinct wordings and made the test
assert on the event line specifically. With that, the sabotage goes red. This
is the second time in this batch that "assert the outcome" hid an ungated fence;
I have started checking every refusal by removing it.

Closes #50

Prepared with AI assistance; reviewed before submission.

The watch writes a diff per revision and a reviewed baseline, and nothing read
them back. `history` walks the committed reports and prints every state a
matching record has held, each with its retrieval date and the citation of the
revision that set it, ending with the baseline.

What it refuses is the substance. A report states the digest of the bytes on
both sides, so a report comparing against bytes no committed report produced
means a revision is missing, and the timeline says so at that point rather than
joining the two ends. Retrieval order is read from the reports, not the
filenames, and dates that run backwards are refused with both named. Every leg
carries diff's three-state parser comparison, which still has no state meaning
"the same parser".

A record no report mentions is listed with one state from the baseline, so "no
result" cannot mean both "no such record" and "a record nobody has moved". A
match term naming a field no record kind has is an error, not a term that
matches nothing.

`diff --jsonl` now repeats both sides' stamps on every change line, so a line
lifted out of a report still says which retrieval it came from and history need
not infer a date from a filename. It is still one object per change.
@ChelseaKR
ChelseaKR merged commit 38a9ade into main Sep 6, 2026
3 checks passed
ChelseaKR added a commit that referenced this pull request Sep 6, 2026
## What was wrong

Found by smoke-testing the four new verbs together on `main` after #65 merged,
which is the check the brief asks for after a burst of merges.

`data/changes/` does not exist in this repository — the watch has not recorded a
revision yet — but `data/parsed/` is committed. In exactly that state:

```
$ ca-tariff-parse history --id smud-r-tod --all
smud-r-tod: no record matched None. That is not the same as a record that has
never changed, which is reported with one state
# History of smud-r-tod

No record matched.
$ echo $?
5
```

Two things wrong with that.

**`--all` asks for nothing by name**, so nothing coming back is a statement
about what has been committed, not a request that failed. Exiting 5 tells a
script the request was wrong when the record is simply empty — the same shape as
the defects this verb was written to avoid, on the other side of the fence.

**`{args.match!r}` renders `None`** when `--all` was used, so the message names a
match nobody made.

## What changed

- `history --all` with no committed reports exits **0** and says which directory
  holds no reports, and that this is a statement about what has been committed
  rather than about whether the document has changed. `--match` selecting
  nothing still exits **5**.
- The empty report body now says that neither an empty match nor an empty report
  set means the document has not changed — it means the committed record does
  not cover it. The old wording, "No record matched.", was the only thing a
  reader saw and it was not true for `--all`.

## How it was verified

Full gate green: `ruff check`, `ruff format --check`, `mypy` (strict), `pytest`,
coverage 93.86% against an 85% floor. Two tests added.

The added test builds the repository's *actual* state — a reviewed baseline
committed with an empty `changes/` beside it — rather than an empty directory,
because with neither present an earlier guard fires first and the test would
have passed without exercising this path at all.

Negative control: reverting the `--all` branch (`if False:`, read back at
`cli.py:324`) turns `test_all_with_no_committed_reports_is_not_a_failed_match`
red. Restored from a pre-edit byte copy (`diff` clean) and re-run green.

Prepared with AI assistance; reviewed before submission.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a history verb that rebuilds a value's timeline from committed watch reports

1 participant