A statement carries rows for reachability, conformance, decision, efficacy,
fault.malformedOutput, fault.scriptMissing, fault.interpreterMissing, fault.timeout,
interference, and the two cost.* rows. A reader meeting a statement for the first time can see
the verdicts without knowing what was actually asked.
What to do: write one short section per attribute — what question it asks, what the producer
does to answer it, and what each verdict means. Concrete: name the actual probe, not a paraphrase
of the attribute's name.
The part that matters most: be explicit about when each attribute reports absence rather
than a bad result, and why those are different. NotApplicable on an artifact that declares no
hooks is not a failure; treating it as one is exactly the misreading this format exists to
prevent. The same distinction is why the row-basis figure draws "not measured this run" in neutral
grey rather than at the pale end of a ramp.
Method: read a real statement under paper/measurements/ alongside the code that produces each
row, and describe what the code does. An attribute's name is a label, not a specification, and
several of them are easy to guess wrong from the name alone.
Sized to an afternoon. Files: docs/.
A statement carries rows for
reachability,conformance,decision,efficacy,fault.malformedOutput,fault.scriptMissing,fault.interpreterMissing,fault.timeout,interference, and the twocost.*rows. A reader meeting a statement for the first time can seethe verdicts without knowing what was actually asked.
What to do: write one short section per attribute — what question it asks, what the producer
does to answer it, and what each verdict means. Concrete: name the actual probe, not a paraphrase
of the attribute's name.
The part that matters most: be explicit about when each attribute reports absence rather
than a bad result, and why those are different.
NotApplicableon an artifact that declares nohooks is not a failure; treating it as one is exactly the misreading this format exists to
prevent. The same distinction is why the row-basis figure draws "not measured this run" in neutral
grey rather than at the pale end of a ramp.
Method: read a real statement under
paper/measurements/alongside the code that produces eachrow, and describe what the code does. An attribute's name is a label, not a specification, and
several of them are easy to guess wrong from the name alone.
Sized to an afternoon. Files:
docs/.