Skip to content

test(engine): gate the catalog against the card that was printed - #830

Merged
ninthworld merged 1 commit into
mainfrom
test/catalog-printed-data-gate
Aug 8, 2026
Merged

ninthworld merged 1 commit into
mainfrom
test/catalog-printed-data-gate

Conversation

@ninthworld

Copy link
Copy Markdown
Owner

Closes #822. Part of #819. Stacked on #829 — review that first; this PR's diff against main will collapse to its own once #829 merges.

build.rs validates a definition's shape. It cannot know a number is wrong, and the behavioural tests assert against whatever the definition says — so they cement the bug rather than catch it. Fifteen cards shipped that way (#820); twelve differed in a field a machine can check.

What lands

  • crates/sage-engine/tests/fixtures/printed_characteristics.json — one record per catalog card: name, mana cost, supertypes, types, subtypes, power, toughness, loyalty, colours. 72 KB.
  • crates/sage-engine/tests/printed_characteristics.rs — five tests: both directions of the catalog↔fixture correspondence, the characteristics themselves, second faces, and the licensing constraint.
  • scripts/printed-characteristics.py — regenerates the fixture from a third-party set file.

The one rule that keeps it honest

The script reads the set file and the catalog's identities (functional_id, name). It never reads a definition's characteristics. A fixture regenerated from the thing it checks is a gate that passes forever and proves nothing — that's the failure mode this is designed against, and it's stated in the script, in the test's module doc, in docs/card-schema.md, and in the engine AGENTS.md.

Falsification

Reverting two of #829's fixes and running the gate:

the catalog disagrees with the card that was printed:
  marauder_s_axe: mana_cost is "{3}", printed "{2}"
  regal_bloodlord: mana_cost is "{3}{B}{B}", printed "{3}{W}{B}"
  regal_bloodlord: subtypes is ["Vampire", "Noble"], printed ["Vampire", "Soldier"]
  regal_bloodlord: colors is [Black], printed [White, Black]

Card and field, named.

Licensing

AGENTS.md forbids shipping rules text, flavour text, art, symbols, or branding. The fixture carries the same functional data ADR 0009 already sources for the catalog itself and nothing else. deny_unknown_fields refuses anything else structurally; a test also states the allowed set outright, because the reason lives in AGENTS.md rather than in serde.

What it does not catch, stated plainly

Rules text is out of scope — it cannot be checked without storing it, and storing it is the one thing the rule forbids. Of #820's fifteen wrong cards this catches twelve. A missing {R} in a cost list, a gain_life the card does not print, a token created tapped: those stay a human-review problem, held by the behavioural tests instead. The module doc says so rather than implying broader coverage.

* is not a number

Enigma Drake's printed power is *. The fixture keeps power as a string, and a * is satisfied by a defined_power ability rather than by a value — so a base of 0 with nothing to define it fails, which is a 0/4 wearing a *'s clothes.

Costs 0.02s in make check. make verify green.

🤖 Generated with Claude Code

Base automatically changed from fix/m19-card-accuracy to main August 8, 2026 01:37
`build.rs` validates a definition's shape — the file stem matches the identity,
a creature carries both power and toughness, a planeswalker carries loyalty and
nothing else does. None of that can know a number is wrong, and the behavioural
tests assert against whatever the definition says, so a mis-transcribed mana
cost is invisible at every gate and then cemented by the test written over it.
Fifteen cards shipped that way (#819); twelve differed in a field a machine can
check.

This is that machine. A fixture transcribed from the printed set, compared to
`CardDatabase::bundled()` — the embedded catalog the game is actually played
with, not the JSON on disk. It names the card and the field when they disagree.

The fixture carries name, mana cost, supertypes, types, subtypes, power,
toughness, loyalty, and colours, and nothing else: the same functional data
ADR 0009 already sources for the catalog, and nothing the licensing rule
forbids. `deny_unknown_fields` refuses anything else structurally, and a test
states the allowed set outright so the reason is not left to serde.

The one rule that keeps this honest is that `scripts/printed-characteristics.py`
reads a third-party set file and the catalog's *identities*, never a
definition's characteristics. A fixture regenerated from the thing it checks is
a gate that passes forever and proves nothing; that is the failure mode this is
designed against, and it is stated where the fixture, the script, and the
engine guide can all be read.

A printed `*` is not a number, so the fixture keeps power as a string: Enigma
Drake's `*` is satisfied by a `defined_power` ability rather than by a value,
and a base of 0 with nothing to define it fails.

Rules text is out of scope, and the limit is real: of the fifteen wrong cards
this catches twelve. A missing {R} in a cost list, a life gain the card does not
print, a token created tapped — those stay a human-review problem, held by the
behavioural tests instead. Costs 0.02s in `make check`.

Closes #822. Part of #819.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ninthworld
ninthworld force-pushed the test/catalog-printed-data-gate branch from 97242d5 to 73f5f64 Compare August 8, 2026 01:37
@ninthworld
ninthworld merged commit 8f7cb6c into main Aug 8, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(engine): gate the catalog against real printed card data

1 participant