Surfaced by run 2 of the validation program (#16) against QuantEcon/meta, 2026-08-25. Record in #56 §3 and §7.
QuantEcon/meta was chosen for run 2 precisely because it is the hardest case for doctrine rule 1 — "the thread is a hypothesis and the default branch is the evidence" — in a repo whose default branch is two files. #16 predicted this would ask what [verified] means when the artifact is a decision, and whether the doctrine needs a fourth evidence class or an explicit exclusion.
The run answered it by improvising. It used [verified live YYYY-MM-DD] for 97 of its 176 evidence tags, checking claims against other repositories' live state, published sites, and workflow files rather than against meta itself — for example confirming that no lecture repo carries a LICENSE file by taking a 404 from GET /repos/{repo}/license across six repos, and that discourse.quantecon.org has no DNS record while five published pages still link to it.
That adaptation is sound and the catalog declares it in its own legend. The problem is that it is improvised per run rather than specified, so the next audit against a code-free tracker may invent a different convention or fall back to [stated] for claims that were in fact checked.
Two things worth settling in references/audit/doctrine.md §2:
Name the class. [verified live <date>] means checked against a live API or published site on that date — inherently not reproducible from a frozen ref, which is exactly why it needs its own tag rather than being folded into [verified]. The date is load-bearing and should be required, since the whole point is that the observation can go stale.
Say when ref-relative verification does not apply. §2 currently requires every [verified] citation to resolve on the named baseline ref. For a tracker whose repository holds no relevant code, that requirement is unsatisfiable rather than merely inconvenient, and an audit that follows it literally would tag genuinely-checked findings as [stated] — understating its own evidence, which is its own kind of dishonesty.
Surfaced by run 2 of the validation program (#16) against
QuantEcon/meta, 2026-08-25. Record in #56 §3 and §7.QuantEcon/metawas chosen for run 2 precisely because it is the hardest case for doctrine rule 1 — "the thread is a hypothesis and the default branch is the evidence" — in a repo whose default branch is two files. #16 predicted this would ask what[verified]means when the artifact is a decision, and whether the doctrine needs a fourth evidence class or an explicit exclusion.The run answered it by improvising. It used
[verified live YYYY-MM-DD]for 97 of its 176 evidence tags, checking claims against other repositories' live state, published sites, and workflow files rather than againstmetaitself — for example confirming that no lecture repo carries aLICENSEfile by taking a 404 fromGET /repos/{repo}/licenseacross six repos, and thatdiscourse.quantecon.orghas no DNS record while five published pages still link to it.That adaptation is sound and the catalog declares it in its own legend. The problem is that it is improvised per run rather than specified, so the next audit against a code-free tracker may invent a different convention or fall back to
[stated]for claims that were in fact checked.Two things worth settling in
references/audit/doctrine.md§2:Name the class.
[verified live <date>]means checked against a live API or published site on that date — inherently not reproducible from a frozen ref, which is exactly why it needs its own tag rather than being folded into[verified]. The date is load-bearing and should be required, since the whole point is that the observation can go stale.Say when ref-relative verification does not apply. §2 currently requires every
[verified]citation to resolve on the named baseline ref. For a tracker whose repository holds no relevant code, that requirement is unsatisfiable rather than merely inconvenient, and an audit that follows it literally would tag genuinely-checked findings as[stated]— understating its own evidence, which is its own kind of dishonesty.