You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Compliance cases declare an expected outcome only for meta-schema validity #158
A case in examples/compliance/ says valid: true or valid: false, and that expectation is checked against the meta-schema only. The other tiers - generated-instance satisfiability, RDF round-trip, pattern lint, predicate attribution - have no per-case expectation at all. They assert "nothing failed" over the corpus as a whole.
What that costs
A rule cannot be pinned. There is no way to write "this document must fail, citing OOLD-RT-d9bd". A fixture can only say the document is well-formed or is not, so most of the catalogue is unexpressible as a test. That is the mechanism behind Conformance fixtures cover a minority of the machine-checkable rules #157.
A wrong pass is invisible. "No failures" is satisfied by a checker that never fires. Several checks in both implementations passed for exactly that reason until a fixture forced them to run.
Agreement substitutes for correctness. With no declared outcome, the only available signal is that two implementations did the same thing, which is also what happens when both are wrong.
Shape
Give a case an expected outcome per tier, keyed to a rule id where one applies: which check, what status, which rule cited. The catalogue already provides stable ids, and coverage.rules already walks the mapping from check to rule, so the vocabulary exists; what is missing is the case format that uses it.
This is the prerequisite for OO-LD/oold-python#148, which replaces verdict comparison with a per-case comparison, and it is what turns the corpus into something a future port can be held to without anyone writing a pairwise harness for it.
A case in
examples/compliance/saysvalid: trueorvalid: false, and that expectation is checked against the meta-schema only. The other tiers - generated-instance satisfiability, RDF round-trip, pattern lint, predicate attribution - have no per-case expectation at all. They assert "nothing failed" over the corpus as a whole.What that costs
OOLD-RT-d9bd". A fixture can only say the document is well-formed or is not, so most of the catalogue is unexpressible as a test. That is the mechanism behind Conformance fixtures cover a minority of the machine-checkable rules #157.Shape
Give a case an expected outcome per tier, keyed to a rule id where one applies: which check, what status, which rule cited. The catalogue already provides stable ids, and
coverage.rulesalready walks the mapping from check to rule, so the vocabulary exists; what is missing is the case format that uses it.This is the prerequisite for OO-LD/oold-python#148, which replaces verdict comparison with a per-case comparison, and it is what turns the corpus into something a future port can be held to without anyone writing a pairwise harness for it.