Found by an adversarial review of PR #110 and deferred out of it, because the answer is a decision rather than a patch.
What is committed today
TRACE_PATTERNS retains the harness's Loading skills from: line, which names absolute paths. The throwaway home sits under os.tmpdir(), so the committed line carries whatever TMPDIR resolved to. From bench/probes/2026-08-07-claude-user-d80e11b7.json, in the repository now:
Loading skills from: managed=/Library/Application Support/ClaudeCode/.claude/skills,
user=/var/folders/k6/m22fb9pj4173x8z2llh_nr480000gn/T/sw-probe-CZdwgh/installed/home/.claude/skills, project=[]
Measured, on darwin. The macOS token is opaque per-user rather than a name, which is why this is a decision and not an incident.
Reasoned, not reproduced: on Windows os.tmpdir() resolves under the user profile, so the same line would commit C:\Users\<name>\AppData\Local\Temp\... into an append-only, published record. Nobody has run the probe on a real Windows host, which bench/probes/README.md already states as an unverified item.
Why nothing catches it
checkRecord scans a whole record for credential shapes and asks nothing else. A path is not credential-shaped.
Promotion into bench/samples/ has the equivalent refusal — AGENTS.md: "every retained file is scanned for operator configuration" — but a probe record is committed by hand and goes through no promotion surface.
Why this is an ADR and not a line
AGENTS.md already settles the shape of this choice for the sample path, and the reasoning transfers:
Redaction is the design's other option and nothing builds it, so the refusal is total until something does.
So the options are to refuse a record whose retained trace carries a home-derived absolute path, or to build redaction and make the probe corpus its second consumer. Refusing is the cheaper and more consistent answer and it costs the Loading skills from: line, which is one of the two lines the trace exists to retain. That is a real loss: it is the line that names WHERE the harness looked, and four documents quote it. Redaction preserves the line and is the thing nothing has built.
bench/probes/README.md now tells a committer to check the line by hand, which is a stopgap and not the answer.
Prior art: ADR-0024, bench/collect-probe.mjs (TRACE_PATTERNS, skillTraceLines), bench/probe.mjs (checkRecord), and the promotion refusals in bench/retain.mjs.
Found by an adversarial review of PR #110 and deferred out of it, because the answer is a decision rather than a patch.
What is committed today
TRACE_PATTERNSretains the harness'sLoading skills from:line, which names absolute paths. The throwaway home sits underos.tmpdir(), so the committed line carries whateverTMPDIRresolved to. Frombench/probes/2026-08-07-claude-user-d80e11b7.json, in the repository now:Measured, on darwin. The macOS token is opaque per-user rather than a name, which is why this is a decision and not an incident.
Reasoned, not reproduced: on Windows
os.tmpdir()resolves under the user profile, so the same line would commitC:\Users\<name>\AppData\Local\Temp\...into an append-only, published record. Nobody has run the probe on a real Windows host, whichbench/probes/README.mdalready states as an unverified item.Why nothing catches it
checkRecordscans a whole record for credential shapes and asks nothing else. A path is not credential-shaped.Promotion into
bench/samples/has the equivalent refusal — AGENTS.md: "every retained file is scanned for operator configuration" — but a probe record is committed by hand and goes through no promotion surface.Why this is an ADR and not a line
AGENTS.md already settles the shape of this choice for the sample path, and the reasoning transfers:
So the options are to refuse a record whose retained trace carries a home-derived absolute path, or to build redaction and make the probe corpus its second consumer. Refusing is the cheaper and more consistent answer and it costs the
Loading skills from:line, which is one of the two lines the trace exists to retain. That is a real loss: it is the line that names WHERE the harness looked, and four documents quote it. Redaction preserves the line and is the thing nothing has built.bench/probes/README.mdnow tells a committer to check the line by hand, which is a stopgap and not the answer.Prior art: ADR-0024,
bench/collect-probe.mjs(TRACE_PATTERNS,skillTraceLines),bench/probe.mjs(checkRecord), and the promotion refusals inbench/retain.mjs.