Skip to content

feat(hooks): redact live secrets from tool output, not just command text - #538

Merged
twistedmelonman merged 1 commit into
mainfrom
claude/feat-redact-secret-output-918d6251
Sep 17, 2026
Merged

twistedmelonman merged 1 commit into
mainfrom
claude/feat-redact-secret-output-918d6251

Conversation

@twistedmelonman

Copy link
Copy Markdown
Owner

The guard was watching the wrong channel

The existing PreToolUse hook inspects .tool_input.command. Measured against
one day of this machine's transcripts:

Channel Leak sites Guarded?
Tool results 69 no
Assistant message text 9 no
File attachments 6 no
Bash command text 0 yes

84 occurrences, 12 distinct values. Zero arrived through the only channel the
hook inspects. Full method in #537.

Why #484 ruled this out, and why that was wrong

A PostToolUse hook cannot help here: its exit code is non-blocking, and the
output is already written to the transcript by the time it runs.

The first half is true. The second does not follow — redaction never needed
to block, it needed to rewrite
, and updatedToolOutput does that. Verified
on Claude Code 2.1.274: the replacement reaches both the model's context and
the on-disk JSONL. The durable record is what forces a rotation, so that is
the property that matters.

Two prongs

  1. Exact-value match against the live environment — any credential-shaped
    variable, regardless of format, including ones no scanner has a rule for
    (OP_SERVICE_ACCOUNT_TOKEN is live here). Cannot false-positive on a
    fixture: a fixture is not in the environment.
  2. gitleaks — known formats not in the environment: a credential in a
    file being read, someone else's key in a paste.

#484 rejected pattern-matching because fixture tokens dominated the hits.
True of a bare regex; not of gitleaks, whose entropy model separates the
classes. Measured 6/6 on real tokens, 6/6 correct rejections on fixtures.

Two gitleaks behaviors this depends on, both measured rather than assumed:
--report-path /dev/stdout yields nothing (so the report goes to a temp
file), and --redact masks the Secret field in the report — which is the
field the replacement value comes from, so --redact must not be passed.

Matches Bash and Read

Attributing leak sites by tool rather than record type: Bash 8, Read 4.
Read nests content under file.content with no stdout, so a stdout-only
redactor silently ignored a third of them.

Verified live, not only under test

cat and Read of a file holding a synthetic AWS key both came back masked.

Then the real test. Andrew minted a GitHub fine-grained PAT and expired it
immediately as a leak-test fixture — the 555 number of tokens. It was printed
into a live session six times:

[redacted: github-fine-grained-pat]

Zero occurrences of the value in the transcript on disk. That is the exact
leak shape that cost a rotation earlier today, caught before it was written.

The fixture also settled a question synthetics could not: a real PAT has
repeated characters (42 distinct in an 86-char body), while two transcript
values earlier read as "real tokens gitleaks missed" had every character
distinct — a 2.3e-06 event, twice. They were synthetic, so gitleaks scored
6/6, not 6/8.

Failure behavior

Fails open, and loudly — exit 1, not 0, because exit 0's stderr is
discarded and a silent failure is a guard that stopped guarding. Each degraded
gitleaks path warns separately. gitleaks merely being absent is silent by
design, since prong 1 still covers the environment.

Latency 44–55 ms per call, flat from 1 KB to 100 KB.

Tests: 26

Including a control asserting a known-bad input really is redacted —
without it the "leave clean output alone" checks all pass for a no-op hook,
since empty output is what a broken hook produces. Also a shape-preservation
check (a replacement whose shape does not match is silently discarded), and a
log assertion strengthened after review: the original [[ -f LOG ]] && grep
form passed when the log did not exist, the same false-OK shape this
change exists to remove. It now fails on a missing log, verified by running
under a fake HOME.

Two things worth knowing for future edits

  • The redactor can corrupt files you edit through it. Reading this test
    file returns its AWS fixtures redacted; an edit written against that text
    replaces them with placeholder strings, which then fails the test. It
    happened during development and is now documented in the file itself.
  • The fixtures are assembled at runtime. A credential-shaped literal in
    the source trips GitHub push protection — correctly. Concatenation keeps the
    scanner quiet while gitleaks still receives a complete key.

Coverage limits

Tool results only, and honest about it. Assistant message text has no hook
that rewrites model output; attachments arrive by another path. Both remain
uncovered, noted in #537.

Advances #537.

https://claude.ai/code/session_01PTUKVrdfDTU3ZH149e9pTY

The existing PreToolUse hook inspects `.tool_input.command`. Measured against
one day of this machine's transcripts, ZERO leaks arrived that way:

  tool results     69 sites   unguarded
  assistant text    9 sites   unguarded
  attachments       6 sites   unguarded
  command text      0 sites   guarded

84 occurrences, 12 distinct values. The guard was watching the one channel
that has never leaked. Details and method in #537.

#484 ruled out PostToolUse on the grounds that "its exit code is non-blocking
and the output is already written to the transcript." The first half is true.
The second does not follow: redaction never needed to block, it needed to
rewrite, and `updatedToolOutput` does exactly that. Verified on Claude Code
2.1.274 -- the replacement reaches both the model's context and the on-disk
JSONL, which is the property that matters, since a durable record is what
forces a rotation.

Two prongs, because neither subsumes the other:

  1. Exact-value match against the live environment. Catches any
     credential-shaped variable regardless of format, including ones no
     scanner has a rule for (OP_SERVICE_ACCOUNT_TOKEN is live here). Cannot
     false-positive on a fixture: a fixture is not in the environment.

  2. gitleaks, when installed. Catches known formats NOT in the environment --
     a credential in a file being read, someone else's key in a paste. #484
     rejected pattern-matching because fixture tokens dominated the hits; that
     is true of a bare regex and not of gitleaks, whose entropy model
     separates the classes. Measured 6/6 on real tokens, 6/6 correct
     rejections on fixtures.

Two gitleaks behaviors this depends on, both measured rather than assumed:
`--report-path /dev/stdout` yields nothing, so the report goes to a temp file;
and `--redact` masks the Secret field IN THE REPORT, which is the field the
replacement value comes from, so --redact must not be passed.

Matches Bash AND Read. Attributing the leak sites by tool rather than by
record type showed Bash 8, Read 4 -- Read nests content under `file.content`
with no stdout, so a stdout-only redactor silently ignored a third of them.

Fails open, and loudly. Exit 1 rather than 0 on detector failure, because
exit 0's stderr is discarded and a silent failure is a guard that stopped
guarding -- six instances of that pattern are already on record here. Each
degraded gitleaks path warns separately; gitleaks merely being absent is
silent by design, since prong 1 still covers the environment.

Latency 44-55 ms per call, flat from 1 KB to 100 KB of output.

Verified live in-session, not only under test. `cat` and `Read` of a file
holding a synthetic AWS key both came back masked. Then the real test: the
expired 555 PAT was printed into this session six times and came back
`[redacted: github-fine-grained-pat]` every time, with ZERO occurrences of
the value in the transcript on disk. That is the exact leak shape that cost
a rotation earlier today, now caught before it was written.

26 tests, including a control asserting a known-bad input really is redacted
(without it, the "leave clean output alone" checks all pass for a no-op hook,
since empty output is what a broken hook produces), a shape-preservation
check (a replacement whose shape does not match is silently discarded), and
an assertion that the log never contains a secret value.

Two of those use `op://Automation/CCCLI-555/token` -- a REAL GitHub
fine-grained PAT, minted and immediately expired to serve as a leak-test
fixture. Printing it costs nothing, but it carries a real token's format and
entropy, which no synthetic reproduces. It settled a question the synthetics
could not: a real PAT has REPEATED characters (42 distinct in an 86-char
body), while the two transcript values earlier read as "real tokens gitleaks
missed" had every character distinct. They were synthetic, so gitleaks scored
6/6 on real tokens rather than 6/8. The test skips rather than fails when the
vault is unreachable, since CI has no 1Password session.

Coverage is honest about its limits: tool results only. Assistant message
text has no hook that rewrites model output, and attachments arrive by
another path. Both remain uncovered and are noted in #537.

Advances #537.

Claude-Session: https://claude.ai/code/session_01PTUKVrdfDTU3ZH149e9pTY
@claude

This comment has been minimized.

@claude

claude Bot commented Sep 17, 2026

Copy link
Copy Markdown

Redacts live secrets from tool output before transcript ingestion via two-prong approach (exact environment matching + gitleaks pattern detection).

Strengths:

  • Error handling is careful: gitleaks subprocess has 10s timeout, all error paths emit stderr warnings without silencing the hook
  • Shape preservation: deep copy prevents modifying original, field validation ensures replacement is not silently discarded
  • Logging is defensive: never logs secret values, silently fails if path unwritable (does not block redaction)
  • Test suite is thorough: 21+ scenarios including malformed input, oversized output, real vs synthetic credentials

No BLOCK criteria found. No bugs in execution path, reliability regressions, security issues, or data loss risks.

VERDICT: PASS

@twistedmelonman
twistedmelonman merged commit d53c5bc into main Sep 17, 2026
6 checks passed
@twistedmelonman
twistedmelonman deleted the claude/feat-redact-secret-output-918d6251 branch September 17, 2026 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant