Code, results, and figures for an anonymized paper under review.
A correspondence audit of dataset documentation: what cards claim vs. what repositories contain, for 1,400 datasets.
- No literally wrong number among 200+ verified claims - and a third still fail verification because nothing binds a number to its referent
- 5 hosted-subset cases: the card describes a corpus the repository only partly hosts
- Ecosystem: 546 discussion threads -> 1 merged fix; 19 clusters of near-duplicate cards; claim-making cards drift 4x less than random ones
- 65% of existing claims would become automatically checkable with anchors as notation
Every number and figure in the paper is produced by a script in code/
from the files in results/. Hand-audit tables (CSV) sit beside the
machine outputs they correct.
code/- numbered, self-documenting pipeline scripts (each docstring states the plain-language question it answers)results/- every result file the paper cites, including audit tables and detector precision/recall recordsfigures/- the paper's figures as generatedEXPERIMENT_LOG.zh.md- the running lab log (Chinese), kept verbatim for provenance
All inputs are public: dataset cards (README raw endpoints), the platform's /size, /splits, /statistics inspection interfaces, commit histories (bare clones), and discussion threads. Scripts re-fetch everything; frozen slugs live in results/.
Python 3.8+; scripts list their own imports. Run scripts in numeric
order; every script is resumable and writes into results/.