Add a test suite to t3-hardware-scoring, and fix what it found

[luckiday]

The skill had no tests. Adding them turned up three bugs that change audit
verdicts, plus a set of documentation that had drifted from the code it
describes.

What the tests found

Both fully worked examples in t3-classification.md disagreed with the script
they document.

Example	Doc says	v2.1 script said
Gray Zone	Toy + Trash (Gray Zone)	Trash (Gray Zone)
Eagle Eye	final_label: "Tool + Trash"	Tool + Trash (Eagle Eye)

The Gray Zone case failed because the Litmus Gate override (Step 3 rule 2) was
documented but never implemented, so resolution fell through to the composite
and picked the opposite category.

A missing litmus_gate was read as "No", silently. The three
*-auditor.md guides show a JSON body — under a heading reading "MUST Follow
This Structure" — with no top-level litmus_gate, while synthesize_results.py
reads exactly that field with a "No" default. An auditor following its own
rubric produced a report whose gate silently read No. The gate is one of only
four conditions deciding the primary classification:

v2.1 : gates read as {tool: No, toy: No, trash: No} -> Low confidence, exit 0
v2.2 : refuses -> 'tool' report is missing the top-level litmus_gate field

An empty critical_issues shadowed a populated one. When the top-level
array was empty but extract_for_report.critical_issues was not, the veto
silently did not fire — turning a vetoed product back into a clean Tool. This is
a safety veto, so disagreement between the two copies now errs toward firing it,
and the validator flags the disagreement.

The Eagle Eye trigger set differed across four files. trash-red-flags.md
defined 14 patterns, trash-auditor.md listed 5, and eagle-eye-validator.md
claimed "the 6 canonical" triggers while omitting Core Flaw — which the auditor
guide did list. The Trash Auditor scored against one list while the Eagle Eye
Validator reviewed its work against another.

What changed

Tests — 136 of them, wired to CI, stdlib-only so there is no install step.

tests/fixtures/golden-ai-pendant/ is a complete worked audit of a fictional
product: Brand-Blinded source text, all three auditor reports, and the
classification they synthesize to. It doubles as the reference for what a
finished report looks like — the rubric guides only ever showed fragments with
... placeholders.

The strongest test is test_every_quote_exists_in_the_source_text: every string
in every verbatim_evidence array must appear in 02-brand-blinded.md. Zero
hallucination is the skill's central claim, and this makes it checked rather
than asserted.

Step 3.5 is a script, not prose. validate_auditor_json.py replaces the
hand-merge instructions: it checks item coverage, that totals equal the sum of
their items, the evidence-first rule, gate consistency, and that every
"Triggered:" item has a matching critical_issues entry — then builds
auditor_reports.json by copying rather than retyping. It refuses to merge on
any error.

One canonical trigger list. trash-red-flags.md now opens with a Canonical
Trigger Index; the other copies are pinned to it by
test_trigger_consistency.py. The duplication is deliberate — an auditor
subagent reads one rubric file and should not need to open a second to know what
fires — so the copies are pinned rather than merged.

Web crawling removed. crawl_product_info.py used requests.get plus regex
HTML stripping, which cannot handle JS-rendered pages; Step 1 now uses the host
agent's own web tools. It also declared beautifulsoup4 as a dependency without
ever importing it. The skill is now stdlib-only.

Also fixed: added the missing references/trash-auditor-template.md, which
trash-auditor.md had pointed at since v2.0; added
.claude-plugin/marketplace.json so the plugin install path the README
documents actually resolves; documented display_label in report-schema.md;
renamed defluff-guide.md, whose central rule is "do NOT delete hype", to
brand-blinding-guide.md; removed the deprecated peer-review-guide.md; and
corrected README references to tooling that was never committed.

final_label now stays clean for leaderboard grouping while display_label
carries the annotations — which also stops (Gray Zone) being dropped whenever
Eagle Eye fired too.

Verification

20 deliberate mutations were introduced to confirm the suite actually detects
breakage — every threshold, the composite direction, the veto, the gate
override, and each validator check. All 20 are caught. Two tests that initially
passed for the wrong reason were tightened. The suite was re-run from a fresh
clone to confirm it depends on nothing untracked.

🤖 Generated with Claude Code