A statistical audit tool that catches label-leaking shortcuts in preference-pair training data (DPO/RLHF-style datasets) before they reach a training run.
In plainer terms: if you're building a dataset with two label classes — say, "the model should change its answer here" vs. "the model should hold its answer here" — it's easy for the two classes to end up separable by surface features (vocabulary, sentence register, a recurring phrase) instead of the actual semantic distinction you want the model to learn. A model trained on data like that can ace your eval by detecting the surface tell instead of doing the real work, and you won't find out it cheated until it fails in the wild. This tool runs a battery of checks — bag-of-words separability, an embedding probe for structural tells BoW misses, an entity-masking control to isolate style from content, a first-person register check, response n-gram dominance detection, and a length-ratio check — and fails loudly, with the specific words or phrases driving the leak named in the output, so you can fix the data instead of guessing.
This came out of a real bug: an early version of a dataset I was building had two classes that a trivial classifier separated with 100% accuracy, because each class had been generated by a different process with a different vocabulary. The model would have learned "detect which prompt generated this" instead of the thing I actually wanted it to learn. This tool is the institutionalized version of the audit that caught it, so the next contamination doesn't require rediscovering the problem from scratch.
from shortcut_gate.probe import run_probe
items = [
{"text": "...", "label": "class_a", "response": "..."},
{"text": "...", "label": "class_b", "response": "..."},
# ...
]
report = run_probe(items)
if not report.passed:
print(report.failures) # names the specific tell, doesn't just say "fail"Or from the command line against a JSONL file (one {"text", "label", "response"}
object per line):
python -m shortcut_gate.probe data/my_dataset.jsonlExit code is nonzero on failure, so this can gate a data pipeline directly.
| Check | Catches |
|---|---|
| Bag-of-words probe | Vocabulary-level separability between classes |
| Embedding probe | Structural/syntactic tells bag-of-words misses |
| Masked-entity control | Whether separability survives stripping numbers/names — if it does, it's stylistic, not content |
| First-person rate gap | A specific register fingerprint (personal narrative vs. abstract argument) that's easy to introduce by accident |
| Response n-gram dominance | A class's responses converging on one repeated phrase ("magic phrase" shortcut) |
| Length ratio | Mean length mismatch between classes, a common confound in preference-pair setups |
python -m shortcut_gate.tests.test_probeIncludes regression tests that reconstruct the actual bugs this tool was built to catch — a disjoint-vocabulary case and a subtler register-fingerprint case — plus a clean-data case to confirm it doesn't false-positive on well-matched data.
MIT.