fix(runtime-node): stop redaction masking GenAI/usage token counts - #17
Conversation
The /token/ sensitive-key pattern is an unanchored substring, so it also masks non-secret usage attributes like gen_ai.usage.*_tokens, prompt_tokens, total_tokens, token_count and max_tokens -- unhelpful for an SDK with llmTracing. Anchoring to /token$/ keeps masking secret token keys (token, access_token, authToken) while leaving usage counts intact, matching the existing anti-over-redaction anchoring for card/author. Value-level token scrubbing still catches token-shaped values. Adds tests.
![autter-dev[bot]](https://avatars.githubusercontent.com/in/2920597?s=60&v=4){kind=small}
There was a problem hiding this comment.
🔴 Autter review in progress — running security, correctness & dependency checks on this PR. Follow live step-by-step progress on the autter/review-gate check in the merge box. Merge is blocked until the gate completes; Autter approves automatically when the review comes back clean, and releases this hold with a neutral review when it finds non-blocking issues.
| /e-?mail/, | ||
| /pass(word|wd|phrase)|^pass$/, | ||
| /token/, | ||
| /token$/, |
There was a problem hiding this comment.
🟠 [ai] Sensitive token-containing attribute keys can bypass redaction — Risk: 65/100
Changing the key pattern from /token/ to /token$/ prevents redaction of sensitive keys that contain token without ending in it, such as session_token_id or token_payload. Preserve GenAI usage-count keys through a narrower allowlist or targeted exception while continuing to redact secret-bearing token keys.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Functions/symbols:
RedactOptions,redactAttributes,makeRedactor,AttrValue,CompiledRedactor,toCaseInsensitive - Dependent files:
@opentelemetry/api - Scopes:
@autter/runtime-node
🛠 AI fix prompt (copy & paste into your coding agent)
Keep broad token redaction for arbitrary secret keys, and explicitly exempt only the approved GenAI/usage count attributes (for example, `input_tokens`, `output_tokens`, `prompt_tokens`, `completion_tokens`, `total_tokens`, `token_count`, and `max_tokens`) using a clear allowlist or key-specific predicate. Add regression tests for token keys with suffixes such as `token_value` and `session_token_id`. Blast radius — if this AI-generated slop ships it cascades to the downstream usage that depends on this file: functions `RedactOptions`, `redactAttributes`, `makeRedactor`, `AttrValue`, `CompiledRedactor`, `toCaseInsensitive`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.
Flagged by Autter security & observability checks.
| /e-?mail/, | ||
| /pass(word|wd|phrase)|^pass$/, | ||
| /token/, | ||
| /token$/, |
There was a problem hiding this comment.
🟠 [ai] Sensitive attributes containing token can bypass redaction — Risk: 75/100
Changing /token/ to /token$/ preserves usage fields such as prompt_tokens and token_count, but also allows keys like token_value and token_string to pass through unmasked. Arbitrary secrets under those keys can therefore be exported in telemetry. Use a narrower allowlist for usage-count keys or retain masking for credential-shaped token keys.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Functions/symbols:
RedactOptions,redactAttributes,makeRedactor,AttrValue,CompiledRedactor,toCaseInsensitive - Dependent files:
@opentelemetry/api - Scopes:
@autter/runtime-node
🛠 AI fix prompt (copy & paste into your coding agent)
Preserve broad token-secret coverage while exempting only the explicitly supported GenAI usage-count keys. Restore matching for token-containing credential names, or add a narrowly defined allow-list for numeric usage attributes such as `gen_ai.usage.input_tokens`, `prompt_tokens`, and `total_tokens`, then add regression tests for `token_value`, `token_string`, `bearerTokenValue`, and other token-bearing secret variants. Blast radius — if this is exploited it cascades to the downstream usage that depends on this file: functions `RedactOptions`, `redactAttributes`, `makeRedactor`, `AttrValue`, `CompiledRedactor`, `toCaseInsensitive`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.
Flagged by Autter security & observability checks.
🚦 Pre-merge checks ·
|
| Check | Status | Explanation |
|---|---|---|
| Missing linked tracker issue | 2 potential issue(s) detected (max risk 50/100): packages/runtime-node/test/redact.test.mjs:127, packages/runtime-node/src/redact.ts:152. | |
| Missing CODEOWNERS reviewer approval | 2 potential issue(s) detected (max risk 50/100): packages/runtime-node/src/redact.ts:145, packages/runtime-node/test/redact.test.mjs:128. |
✅ Passed checks (167)
| Check | Status | Explanation |
|---|---|---|
| Too many files changed | ✅ Passed | Changed 2 file(s), within the limit of 50. |
| Too many lines changed | ✅ Passed | Changed 90 line(s), within the limit of 1000. |
| Too many unrelated chapters | ✅ Passed | 1 chapter(s) detected, within the limit of 6. |
| Generated files hiding real changes | ✅ Passed | Generated-file volume (0 lines) does not obscure the 90 hand-written line(s). |
| Missing PR context | ✅ Passed | PR context looks sufficient. |
| Mixed concerns (refactor + behavior change) | ✅ Passed | The PR contains a focused redaction behavior change with regression tests, not a refactor combined with behavior work. |
| Migration + app logic + UI combined in one PR | ✅ Passed | The PR contains no database migrations and no user-interface changes. |
| Sensitive data in logs | ✅ Passed | No sensitive data in logs issues detected. |
| Log injection | ✅ Passed | No log injection issues detected. |
| Missing audit logging | ✅ Passed | No missing audit logging issues detected. |
| Removed observability | ✅ Passed | No removed observability issues detected. |
| Silent exception swallowing | ✅ Passed | No silent exception swallowing issues detected. |
| Unhandled promise rejection | ✅ Passed | No unhandled promise rejection issues detected. |
| Circuit breaker not detected | ✅ Passed | No circuit breaker not detected issues detected. |
| Stack trace leakage | ✅ Passed | No stack trace leakage issues detected. |
| Multi-write without detected transaction | ✅ Passed | No multi-write without detected transaction issues detected. |
| Possible TOCTOU in critical path | ✅ Passed | No possible toctou in critical path issues detected. |
| Idempotency key not detected | ✅ Passed | No idempotency key not detected issues detected. |
| Possible non-atomic read-modify-write | ✅ Passed | No possible non-atomic read-modify-write issues detected. |
| Optimistic locking not detected | ✅ Passed | No optimistic locking not detected issues detected. |
| Rate limiting not detected | ✅ Passed | No rate limiting not detected issues detected. |
| Rate limiting removed | ✅ Passed | No rate limiting removed issues detected. |
| Batch size limit not detected | ✅ Passed | No batch size limit not detected issues detected. |
| Pagination not detected | ✅ Passed | No pagination not detected issues detected. |
| Publicly exposed storage | ✅ Passed | No publicly exposed storage issues detected. |
| Over-permissive IAM policy | ✅ Passed | No over-permissive iam policy issues detected. |
| Security group open to the internet | ✅ Passed | No security group open to the internet issues detected. |
| Unencrypted storage at rest | ✅ Passed | No unencrypted storage at rest issues detected. |
| Infrastructure missing access logging | ✅ Passed | No infrastructure missing access logging issues detected. |
| Hardcoded secret in IaC | ✅ Passed | No hardcoded secret in iac issues detected. |
| Infrastructure misconfiguration | ✅ Passed | No infrastructure misconfiguration issues detected. |
| Deprecated Kubernetes API version | ✅ Passed | No deprecated kubernetes api version issues detected. |
| Compound IaC attack chain | ✅ Passed | No compound iac attack chain issues detected. |
| Prompt injection risk | ✅ Passed | No LLM/AI-integration code touched by this diff. |
| LLM output used in a dangerous sink | ✅ Passed | No LLM/AI-integration code touched by this diff. |
| Sensitive data in prompt or system-prompt leakage | ✅ Passed | No LLM/AI-integration code touched by this diff. |
| Over-privileged LLM tool / excessive agency | ✅ Passed | No LLM/AI-integration code touched by this diff. |
| Missing validation on an LLM-driven decision | ✅ Passed | No LLM/AI-integration code touched by this diff. |
| Unbounded LLM usage (denial-of-wallet) | ✅ Passed | No LLM/AI-integration code touched by this diff. |
| Table exposed without row-level security | ✅ Passed | No row-level-security-related code touched by this diff. |
| Over-broad row-level security policy | ✅ Passed | No row-level-security-related code touched by this diff. |
| Code path that bypasses row-level security | ✅ Passed | No row-level-security-related code touched by this diff. |
| Privileged database credential reachable from the client | ✅ Passed | No row-level-security-related code touched by this diff. |
| Privileged query without row-level scoping | ✅ Passed | No row-level-security-related code touched by this diff. |
| Template-default gradient styling | ✅ Passed | No added frontend pages or design-slop markers in this diff. |
| Interchangeable AI marketing copy | ✅ Passed | No added frontend pages or design-slop markers in this diff. |
| Placeholder content shipped to users | ✅ Passed | No added frontend pages or design-slop markers in this diff. |
| Emoji standing in for an icon system | ✅ Passed | No added frontend pages or design-slop markers in this diff. |
| Call-to-action that goes nowhere | ✅ Passed | No added frontend pages or design-slop markers in this diff. |
| Templated page composition | ✅ Passed | No added frontend pages or design-slop markers in this diff. |
| Merge-blocking marker left in the change | ✅ Passed | No pending-work markers added by this diff. |
| Known-defect marker shipped in code | ✅ Passed | No pending-work markers added by this diff. |
| Untracked TODO without an issue reference | ✅ Passed | No pending-work markers added by this diff. |
| Test disabled or left pending | ✅ Passed | No pending-work markers added by this diff. |
| PII in logs | ✅ Passed | No pii in logs issues detected. |
| PII or internals leaked in error response | ✅ Passed | No pii or internals leaked in error response issues detected. |
| PII stored without application-level encryption | ✅ Passed | No pii stored without application-level encryption issues detected. |
| User data stored without retention controls | ✅ Passed | No user data stored without retention controls issues detected. |
| PII sent to external / cross-border destination | ✅ Passed | No pii sent to external / cross-border destination issues detected. |
| Lockfile resolution / integrity tampered | ✅ Passed | No lockfile resolution / integrity tampered issues detected. |
| Dependency runs install-time lifecycle script | ✅ Passed | No dependency runs install-time lifecycle script issues detected. |
| Possible dependency-confusion attack | ✅ Passed | No possible dependency-confusion attack issues detected. |
| Lockfile resolves a dependency the manifest does not declare | ✅ Passed | No lockfile resolves a dependency the manifest does not declare issues detected. |
| Checked-in build artefact modified without source change | ✅ Passed | No checked-in build artefact modified without source change issues detected. |
| Dockerfile build-step is insecure | ✅ Passed | No dockerfile build-step is insecure issues detected. |
| External artefact pulled in without integrity pinning | ✅ Passed | No external artefact pulled in without integrity pinning issues detected. |
| Changed export, importer not updated | ✅ Passed | No changed export with an un-updated importer detected. |
| Missing security-team review on sensitive path | ✅ Passed | No missing security-team review on sensitive path issues detected. |
| Source changes without matching tests | ✅ Passed | No source changes without matching tests issues detected. |
| Migration missing rollback / down step | ✅ Passed | No migration missing rollback / down step issues detected. |
| Frontend importing database client directly | ✅ Passed | No frontend importing database client directly issues detected. |
| Route handler bypassing service layer | ✅ Passed | No route handler bypassing service layer issues detected. |
| Backend service importing UI module | ✅ Passed | No backend service importing ui module issues detected. |
| Cross-context internals import | ✅ Passed | No cross-context internals import issues detected. |
| Workspace package rule violation | ✅ Passed | No workspace package rule violation issues detected. |
| Inconsistent logging pattern | ✅ Passed | No inconsistent logging pattern issues detected. |
| Inconsistent error handling | ✅ Passed | No inconsistent error handling issues detected. |
| Endpoint missing input validation | ✅ Passed | No endpoint missing input validation issues detected. |
| Multi-write without transaction wrapper | ✅ Passed | No multi-write without transaction wrapper issues detected. |
| New feature shipped without feature flag | ✅ Passed | No new feature shipped without feature flag issues detected. |
| Module placed in the wrong workspace package | ✅ Passed | No module placed in the wrong workspace package issues detected. |
| Direct env-var access bypasses config module | ✅ Passed | No direct env-var access bypasses config module issues detected. |
| Hallucinated import (package not installed) | ✅ Passed | No hallucinated import (package not installed) issues detected. |
| Nonexistent package (not found in registry) | ✅ Passed | No nonexistent package (not found in registry) issues detected. |
| Call to function that does not exist | ✅ Passed | No call to function that does not exist issues detected. |
| Generic placeholder identifier in production logic | ✅ Passed | No generic placeholder identifier in production logic issues detected. |
| Repetitive boilerplate (duplicated block) | ✅ Passed | No repetitive boilerplate (duplicated block) issues detected. |
| Overbroad try/catch swallowing all exceptions | ✅ Passed | No overbroad try/catch swallowing all exceptions issues detected. |
| TODO / FIXME on critical path | ✅ Passed | No todo / fixme on critical path issues detected. |
| Comment contradicts or fabricates code behaviour | ✅ Passed | No comment contradicts or fabricates code behaviour issues detected. |
| Abstraction defined but never used | ✅ Passed | No abstraction defined but never used issues detected. |
| Code style differs from rest of codebase | ✅ Passed | No code style differs from rest of codebase issues detected. |
| Established pattern ignored | ✅ Passed | No established pattern ignored issues detected. |
| Unhandled edge case (null / empty / zero / boundary) | ✅ Passed | No unhandled edge case (null / empty / zero / boundary) issues detected. |
| Doc-copy code with insecure defaults | ✅ Passed | No doc-copy code with insecure defaults issues detected. |
| Dead code (defined but never referenced) | ✅ Passed | No dead code (defined but never referenced) issues detected. |
| Deprecated API call | ✅ Passed | No deprecated api call issues detected. |
| API pattern from wrong library version | ✅ Passed | No api pattern from wrong library version issues detected. |
| API endpoint removed | ✅ Passed | No api endpoint removed issues detected. |
| HTTP method changed (GET ↔ POST etc.) | ✅ Passed | No http method changed (get ↔ post etc.) issues detected. |
| New required field added to request | ✅ Passed | No new required field added to request issues detected. |
| Field removed from response schema | ✅ Passed | No field removed from response schema issues detected. |
| Response field type changed | ✅ Passed | No response field type changed issues detected. |
| HTTP status code changed | ✅ Passed | No http status code changed issues detected. |
| Auth requirement added / removed / changed | ✅ Passed | No auth requirement added / removed / changed issues detected. |
| Error response shape changed | ✅ Passed | No error response shape changed issues detected. |
| Pagination behaviour changed | ✅ Passed | No pagination behaviour changed issues detected. |
| Outbound webhook payload schema changed | ✅ Passed | No outbound webhook payload schema changed issues detected. |
| GraphQL field removed without deprecation | ✅ Passed | No graphql field removed without deprecation issues detected. |
| GraphQL enum value removed | ✅ Passed | No graphql enum value removed issues detected. |
| SQL injection | ✅ Passed | No sql injection issues detected. |
| Cross-site scripting (XSS) | ✅ Passed | No cross-site scripting (xss) issues detected. |
| Path traversal | ✅ Passed | No path traversal issues detected. |
| Command injection | ✅ Passed | No command injection issues detected. |
| Insecure deserialization | ✅ Passed | No insecure deserialization issues detected. |
| Weak cryptography | ✅ Passed | No weak cryptography issues detected. |
| Hardcoded secret | ✅ Passed | No hardcoded secret issues detected. |
| Insecure randomness for security material | ✅ Passed | No insecure randomness for security material issues detected. |
| Unsafe file upload | ✅ Passed | No unsafe file upload issues detected. |
| Missing input validation | ✅ Passed | No missing input validation issues detected. |
| Unsafe CORS configuration | ✅ Passed | No unsafe cors configuration issues detected. |
| Unsafe / open redirect | ✅ Passed | No unsafe / open redirect issues detected. |
| Missing CSRF protection | ✅ Passed | No missing csrf protection issues detected. |
| Unsafe cookie / session settings | ✅ Passed | No unsafe cookie / session settings issues detected. |
| Sensitive data exposure | ✅ Passed | No sensitive data exposure issues detected. |
| API key in source | ✅ Passed | No api key in source detected. |
| Access token in source | ✅ Passed | No access token in source detected. |
| Private key in source | ✅ Passed | No private key in source detected. |
| Database connection URL with embedded credentials | ✅ Passed | No database connection url with embedded credentials detected. |
| Cloud credential in source | ✅ Passed | No cloud credential in source detected. |
| Webhook signing secret in source | ✅ Passed | No webhook signing secret in source detected. |
| OAuth client secret in source | ✅ Passed | No oauth client secret in source detected. |
| JWT signing secret in source | ✅ Passed | No jwt signing secret in source detected. |
| Hardcoded password | ✅ Passed | No hardcoded password detected. |
| Auth middleware removed from route | ✅ Passed | No auth middleware removed from route issues detected. |
| Route protection changed (protected → public) | ✅ Passed | No route protection changed (protected → public) issues detected. |
| Permission / RBAC check removed | ✅ Passed | No permission / rbac check removed issues detected. |
| Required role weakened | ✅ Passed | No required role weakened issues detected. |
| Admin-only route exposed to lower privilege | ✅ Passed | No admin-only route exposed to lower privilege issues detected. |
| Token validation skipped in middleware chain | ✅ Passed | No token validation skipped in middleware chain issues detected. |
| JWT verification weakened or changed | ✅ Passed | No jwt verification weakened or changed issues detected. |
| Session expiration / TTL changed | ✅ Passed | No session expiration / ttl changed issues detected. |
| Password reset flow changed | ✅ Passed | No password reset flow changed issues detected. |
| OAuth callback / redirect handling changed | ✅ Passed | No oauth callback / redirect handling changed issues detected. |
| Webhook endpoint missing signature verification | ✅ Passed | No webhook endpoint missing signature verification issues detected. |
| Public route touches private/PII data | ✅ Passed | No public route touches private/pii data issues detected. |
| Frontend performance issue | ✅ Passed | No additional explanation was reported. |
| Frontend security issue | ✅ Passed | No additional explanation was reported. |
| Frontend correctness issue | ✅ Passed | No additional explanation was reported. |
| Accessibility issue | ✅ Passed | No additional explanation was reported. |
| Frontend maintainability issue | ✅ Passed | No additional explanation was reported. |
| Code correctness issue | ✅ Passed | No additional explanation was reported. |
| Runtime error risk | ✅ Passed | No additional explanation was reported. |
| Resource leak risk | ✅ Passed | No additional explanation was reported. |
| Data integrity risk | ✅ Passed | No additional explanation was reported. |
| Maintainability issue | ✅ Passed | No additional explanation was reported. |
| Co-change coupling | ✅ Passed | No additional explanation was reported. |
| Redundant alias / duplicate import | ✅ Passed | No additional explanation was reported. |
| Redundant type construct | ✅ Passed | No additional explanation was reported. |
| Simplifiable code | ✅ Passed | No additional explanation was reported. |
| Unnecessary type assertion | ✅ Passed | No additional explanation was reported. |
| Module smell | ✅ Passed | No additional explanation was reported. |
| Excessive complexity | ✅ Passed | No additional explanation was reported. |
| Dead export (no callers) | ✅ Passed | No additional explanation was reported. |
| Code duplication / DRY violation | ✅ Passed | No additional explanation was reported. |
| Complexity Guard | ✅ Passed | No additional explanation was reported. |
| Bundle Size Monitor | ✅ Passed | No additional explanation was reported. |
This comment is updated automatically whenever Autter reviews a new PR revision.
🧭 PR hygiene & process suggestionsAutter has 2 suggestion(s) about the shape of this PR (size, scope, reviewability). These are process guidance — not code defects — so they are consolidated here instead of posted as inline comments on individual files. 🟠 Missing linked tracker issue — Risk: 50/100This PR's title and body do not reference any tracker issue (GitHub 🛠 AI fix prompt (copy & paste into your coding agent)🟡 Missing CODEOWNERS reviewer approval — Risk: 40/100The added redaction regression tests have no demonstrated approving CODEOWNER review; the only listed reviewer requested changes. This leaves coverage for redactAttributes and its secret-token masking contract unapproved. Blast radius — skipping this guardrail cascades to the downstream usage that depends on this file: scopes 🛠 AI fix prompt (copy & paste into your coding agent)Flagged by Autter PR-hygiene checks.
|
There was a problem hiding this comment.
Autter completed PR review for #17: 5 finding(s) remain below the merge-blocking bar, so this review stays neutral rather than approving. (Also detected: 2 finding(s) dismissed as likely false positives by verification.) See the findings below; the task checklist follows as the review's final comment.
| /e-?mail/, | ||
| /pass(word|wd|phrase)|^pass$/, | ||
| /token/, | ||
| /token$/, |
There was a problem hiding this comment.
🔴 [ai] Narrowed token key matching allows custom secrets to be exported — Risk: 86/100
Changing the built-in token rule from a substring match to /token$/ means sensitive custom attribute names such as token_value, tokenString, or token_id are no longer masked by key. Their values can be arbitrary secrets that do not match the separate JWT/provider-token value regexes, so redactWith copies them unchanged and the server attaches those attributes to spans that are sent to the authenticated OTLP exporter. This is a reachable data-exposure regression for applications passing token-bearing custom attributes, and the added tests only cover token names that still end exactly in 'token'.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Dependent files:
packages/runtime-node/src/redact.ts,packages/runtime-node/src/server.ts
🛠 AI fix prompt (copy & paste into your coding agent)
Preserve masking for token-bearing secret keys while exempting only the documented usage/count attributes, for example by explicitly allowing known numeric GenAI usage keys or by using a boundary-aware token rule rather than requiring the whole key to end in token. Add regression tests for token_value, tokenString, token_id, and id_token alongside the usage-count exemptions.
Flagged by Autter security & observability checks.
| /e-?mail/, | ||
| /pass(word|wd|phrase)|^pass$/, | ||
| /token/, | ||
| /token$/, |
There was a problem hiding this comment.
🔴 [ai] Narrowed token-key matching allows opaque credentials into error telemetry — Risk: 86/100
This change stops treating any key containing token as sensitive and only masks keys ending exactly in token. As a result, common opaque credential names such as tokenValue, session_token_value, or idTokenValue are no longer masked; the value-level rules only recognize specific JWT/provider/bearer formats and will not reliably catch arbitrary credential strings. When application code captures an exception or error message with one of these attributes, the active capture path passes it through activeRedactor and creates the export span, so an exporter can receive the credential in failure telemetry instead of the operation failing safely. The new tests cover only token, access_token, refresh_token, and authToken, leaving this regression unproved for non-suffix token keys.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Dependent files:
packages/runtime-node/src/redact.ts,packages/runtime-node/src/server.ts,packages/runtime-node/test/redact.test.mjs
🛠 AI fix prompt (copy & paste into your coding agent)
Preserve the GenAI usage-count exception without dropping broad secret-key protection: use an explicit allowlist for known usage/count keys or a key-pattern rule that masks token-bearing keys unless they are recognized numeric usage attributes, and add tests for opaque names such as tokenValue and session_token_value through captureException/captureMessage.
Flagged by Autter security & observability checks.
| /e-?mail/, | ||
| /pass(word|wd|phrase)|^pass$/, | ||
| /token/, | ||
| /token$/, |
There was a problem hiding this comment.
🔴 [ai] Token-containing secret keys no longer get masked unless they end exactly in token — Risk: 82/100
Changing the matcher from /token/ to /token$/ makes the redactor miss reachable secret-bearing attribute names such as tokenValue, oauth_token_value, or id_token_hash. These are accepted as arbitrary custom attributes and flow through activeRedactor in llmBaseAttributes (and the capture paths) without any other key-level guard; unless the value happens to match one of the separate token-shape regexes, it is exported unchanged. This contradicts the documented guarantee that attributes whose key looks like token are masked and regresses the prior behavior while fixing usage-count keys that contain tokens.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Dependent files:
packages/runtime-node/src/redact.ts,packages/runtime-node/src/server.ts,packages/runtime-node/README.md
🛠 AI fix prompt (copy & paste into your coding agent)
Preserve the GenAI usage-count exception without weakening token-key redaction: use a matcher that recognizes token as a sensitive key component while explicitly excluding the documented usage/count forms (including `gen_ai.usage.*_tokens`, `prompt_tokens`, `completion_tokens`, `total_tokens`, `token_count`, and `max_tokens`), or otherwise classify usage keys before applying the generic token rule. Add regression coverage for a non-token-shaped secret under `tokenValue` or `oauth_token_value`.
Flagged by Autter security & observability checks.
| /e-?mail/, | ||
| /pass(word|wd|phrase)|^pass$/, | ||
| /token/, | ||
| /token$/, |
There was a problem hiding this comment.
🟠 [ai] Suffix-only token matching leaks credential-bearing token attributes — Risk: 78/100
Changing the sensitive-key rule from substring matching to /token$/ removes the redaction guarantee for custom attribute names where token is followed by a qualifier, such as token_value, api_token_value, or id_token_hint. Those names are accepted through the public Attributes contract and their values can be arbitrary secrets; the new rule sends them through value redaction instead of masking the whole value, and ordinary secret values that do not match the token-shape regexes can therefore be exported. The call chain is active server capture -> activeRedactor -> redactWith -> isSensitiveKey, so this is a production privacy regression rather than a test-only behavior change.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Dependent files:
packages/runtime-node/src/redact.ts,packages/runtime-node/src/server.ts
🛠 AI fix prompt (copy & paste into your coding agent)
Preserve masking for credential-oriented token key variants while exempting known usage-count keys explicitly, or use a narrowly defined token-key regex that handles separators and suffix qualifiers without masking GenAI usage/count attributes.
Flagged by Autter security & observability checks.
There was a problem hiding this comment.
@autter then is it not correct that it should allow all forms of token?
There was a problem hiding this comment.
Not all forms. The intended behavior is to allow the known usage-count keys, but still mask arbitrary credential-bearing names. In this codebase, generated GenAI usage is already added separately in packages/runtime-node/src/server.ts:533-547, while caller-supplied LLM attributes go through the shared redactor at packages/runtime-node/src/server.ts:517-530 and that redactor is also used for captured errors at packages/runtime-node/src/server.ts:813-825. With /token$/ in packages/runtime-node/src/redact.ts:43-50, names such as token_value, tokenString, or id_token_hint can now pass through if their values are opaque secrets that do not match the value regexes. So the safer fix is an explicit allowlist/exception for the usage metrics (for example gen_ai.usage.*_tokens, prompt_tokens, completion_tokens, total_tokens, token_count, max_tokens) while retaining broad token-key redaction for custom attributes. The current tests only prove the suffix cases in packages/runtime-node/test/redact.test.mjs:128-156; they do not cover those credential-bearing variants.
🤖 Replying to your mention — mention @autter again to continue.
🧪 Autter test runAutter checked Execution summary: 4 checks executed · 4 passed. Project test commands
Autter targeted verification4 tests executed · 4 passed. Declared tests: 8 test file(s) found — 0 ran, 0 not observed in suite output, 8 did not run.
|
| Changed file | Related test | Result |
|---|---|---|
packages/runtime-node/src/redact.ts |
packages/runtime-node/test/redact.test.mjs |
✅ existing test passes |
🤖 Coverage-check evidence
packages/runtime-node/src/redact.ts
Ran: npm run build -w @autter/runtime-node && node --test packages/runtime-node/test/redact.test.mjs
TAP: 1..15; # tests 15; # pass 15; # fail 0; build succeeded (ESM/CJS/DTS). Tests 13-15 covered canonical GenAI usage counts and token-key redaction.
Temporary tests are written under .autter/scratch/ for verification only — they are never committed to the repository.
Test plan (from the PR description)
- ✅ Run the runtime-node redaction test suite and verify the new GenAI usage-count assertions pass. — verified by agent execution
- ✅ Run
npm run buildfrom the repository root. — verified by agent execution - ✅ Inspect redacted output for numeric GenAI token counts, string token values, and unrelated keys such as
discard_countandauthor_id. — verified by agent execution
🤖 Agent-executed checks
✅ Run the runtime-node redaction test suite and verify the new GenAI usage-count assertions pass.
Ran: npm install --include=dev && npm test -w @autter/runtime-node
TAP: 1..20; tests 20; suites 0; pass 20; fail 0. Subtests 18 and 19 ('keeps canonical GenAI usage counts only when numeric' and 'redacts non-numeric canonical GenAI usage values') both reported ok.
✅ Run npm run build from the repository root.
Ran: npm run build
Build succeeded for @autter/runtime-browser, @autter/runtime-node, @autter/otlp-ingester, and @autter/runtime-next; all tsup/tsc commands completed successfully.
✅ Inspect redacted output for numeric GenAI token counts, string token values, and unrelated keys such as discard_count and author_id.
Ran: node .autter/scratch/inspect-redaction.mjs
Observed: {"gen_ai.usage.input_tokens":512,"gen_ai.usage.output_tokens":128,"gen_ai.usage.total_tokens":640,"gen_ai.usage.input_tokens_string":"[redacted]","discard_count":7,"author_id":"u_8f2k1"}. Assertions passed for numeric counts, string redaction, and unrelated keys.
⬜ items could not be verified automatically and still need a manual check.
There was a problem hiding this comment.
🔴 Autter review in progress — running security, correctness & dependency checks on this PR. Follow live step-by-step progress on the autter/review-gate check in the merge box. Merge is blocked until the gate completes; Autter approves automatically when the review comes back clean, and releases this hold with a neutral review when it finds non-blocking issues.
| return r.keyPatterns.some((re) => re.test(lowered)); | ||
| const lowered = key.toLowerCase(); | ||
|
|
||
| // Known GenAI usage-count attributes are safe to keep. |
There was a problem hiding this comment.
🟡 [ai] Comment inaccurately categorizes max_tokens as a usage-count attribute — Risk: 10/100
The allowlist includes max_tokens, which is typically a generation-limit/request parameter, not a usage count. Update the comment to describe these as token-related GenAI attributes, or remove max_tokens if only usage counts should bypass redaction.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Functions/symbols:
redactAttributes,redactValue,isSensitiveKey,redactWith,makeRedactor,redactString - Dependent files:
@opentelemetry/api - Scopes:
@autter/runtime-node
🛠 AI fix prompt (copy & paste into your coding agent)
Clarify the comment to describe all allowlisted token-related attributes, or remove `max_tokens` from this set if only usage-count attributes should bypass redaction. Blast radius — if this AI-generated slop ships it cascades to the downstream usage that depends on this file: functions `redactAttributes`, `redactValue`, `isSensitiveKey`, `redactWith`, `makeRedactor`, `redactString`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.
Flagged by Autter security & observability checks.
| "completion_tokens", | ||
| "total_tokens", | ||
| "token_count", | ||
| "max_tokens", |
There was a problem hiding this comment.
🟠 [ai] Sensitive max_tokens attributes bypass configured redaction — Risk: 75/100
The unconditional max_tokens allowlist entry skips all configured key-redaction patterns, so callers that treat this key as sensitive receive its value unredacted. Allowlist only actual usage-count fields or apply configured redaction patterns first.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Functions/symbols:
redactAttributes,redactValue,isSensitiveKey,redactWith,makeRedactor,redactString - Dependent files:
@opentelemetry/api - Scopes:
@autter/runtime-node
🛠 AI fix prompt (copy & paste into your coding agent)
Remove `max_tokens` from the unconditional usage-token allowlist, or only exempt it when it is an explicitly recognized GenAI attribute and not when it matches a configured sensitive-key pattern. Add a regression test proving configured redaction still applies to `max_tokens`. Blast radius — if this defect reaches production it can fail the downstream usage that depends on this file: functions `redactAttributes`, `redactValue`, `isSensitiveKey`, `redactWith`, `makeRedactor`, `redactString`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.
Flagged by Autter security & observability checks.
| const lowered = key.toLowerCase(); | ||
|
|
||
| // Known GenAI usage-count attributes are safe to keep. | ||
| const usageKey = lowered.split(".").pop() ?? lowered; |
There was a problem hiding this comment.
🟠 [ai] Dotted attributes with allowlisted token suffixes bypass redaction — Risk: 75/100
Using lowered.split(".").pop() makes any attribute whose final segment is an allowlisted usage name bypass configured key redaction, including sensitive names such as credentials.input_tokens. Restrict the exemption to recognized GenAI usage attributes rather than arbitrary dotted keys.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Functions/symbols:
redactAttributes,redactValue,isSensitiveKey,redactWith,makeRedactor,redactString - Dependent files:
@opentelemetry/api - Scopes:
@autter/runtime-node
🛠 AI fix prompt (copy & paste into your coding agent)
Match only the explicitly supported GenAI usage attribute names (including their documented full dotted forms), rather than allowing any key whose final segment is in `USAGE_TOKEN_KEYS`. Add regression tests showing unrelated dotted keys such as `credentials.input_tokens` remain redacted. Blast radius — if this defect reaches production it can fail the downstream usage that depends on this file: functions `redactAttributes`, `redactValue`, `isSensitiveKey`, `redactWith`, `makeRedactor`, `redactString`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.
Flagged by Autter security & observability checks.
|
|
|
||
| // Known GenAI usage-count attributes are safe to keep. | ||
| const usageKey = lowered.split(".").pop() ?? lowered; | ||
| if (USAGE_TOKEN_KEYS.has(usageKey)) return false; |
There was a problem hiding this comment.
🔴 [ai] Usage-key exemption bypasses configured sensitive-key redaction — Risk: 86/100
The new suffix allowlist returns false before evaluating every entry in r.keyPatterns. This changes the contract for all callers that pass additionalKeyPatterns: a caller configuring a sensitive key such as total_tokens now receives its value unchanged, despite RedactOptions documenting that matching additional key patterns mask the whole value. The same bypass also affects nested objects because redactValue calls isSensitiveKey for nested keys. Since makeSafeCapture and the server's cached activeRedactor both route caller attributes through this helper, sensitive values can cross those capture chains unmasked; the exemption must only apply to the built-in usage-key case without overriding explicit configured patterns.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Dependent files:
packages/runtime-node/src/server.ts,packages/runtime-node/src/redact.ts
🛠 AI fix prompt (copy & paste into your coding agent)
Preserve the usage-count exception only for keys that are not matched by caller-supplied additionalKeyPatterns (and ensure the same precedence applies to nested keys). Evaluate explicit sensitive patterns before allowing the known usage suffixes, or otherwise distinguish the built-in token rule from configured rules.
Flagged by Autter security & observability checks.
|
|
||
| // Known GenAI usage-count attributes are safe to keep. | ||
| const usageKey = lowered.split(".").pop() ?? lowered; | ||
| if (USAGE_TOKEN_KEYS.has(usageKey)) return false; |
There was a problem hiding this comment.
🔴 [ai] Usage-name exemption bypasses sensitive-key masking for arbitrary namespaces — Risk: 86/100
The new early return treats any key whose final dot segment is a known usage name as non-sensitive before evaluating the built-in or additional key patterns. Consequently a reachable public call such as captureException(error, {"password.token_count": "hunter2"}) or redactAttributes({"api_key.max_tokens": "secret"}) returns the secret unchanged: the key ends in token_count/max_tokens, so the /password/ or /api...key/ rule is never consulted, and the value is not otherwise scrubbed. The intended exemption is for numeric GenAI usage counts, but this implementation exempts arbitrary keys and value types, breaking the documented guarantee that attributes whose names look sensitive are masked before export.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Dependent files:
packages/runtime-node/src/redact.ts,packages/runtime-node/src/server.ts
🛠 AI fix prompt (copy & paste into your coding agent)
Restrict the usage-count exemption to the intended GenAI/usage attribute namespace and numeric usage values, or otherwise ensure built-in and additional sensitive-key patterns take precedence; add regression coverage for sensitive keys ending in token_count and max_tokens.
Flagged by Autter security & observability checks.
|
|
||
| // Known GenAI usage-count attributes are safe to keep. | ||
| const usageKey = lowered.split(".").pop() ?? lowered; | ||
| if (USAGE_TOKEN_KEYS.has(usageKey)) return false; |
There was a problem hiding this comment.
🔴 [ai] Usage-key exemption bypasses secret redaction on captured failures — Risk: 86/100
The early return treats any attribute whose final dot-separated component is a usage token name as safe, before evaluating the built-in or caller-supplied sensitive-key patterns. Consequently a failed request reported with a key such as api.input_tokens or an explicitly configured additionalKeyPatterns: ["input_tokens"] can export its raw secret value; a string value also bypasses value scrubbing because redactWith skips redactValue for the exempt key. The same redactor is used for captured exceptions and error messages before their spans are exported (server.ts:719, server.ts:824, server.ts:868), so the failure path can leak credentials precisely when diagnostic telemetry is emitted. Restrict the exemption to the exact documented GenAI usage attribute names, and still let explicit additional key patterns take precedence.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Dependent files:
packages/runtime-node/src/redact.ts,packages/runtime-node/src/server.ts
🛠 AI fix prompt (copy & paste into your coding agent)
Make the usage exemption apply only to the exact supported GenAI usage attribute names (or otherwise preserve the documented redaction precedence), and ensure caller-provided additionalKeyPatterns can never be bypassed. Add coverage for a secret-valued `api.input_tokens` and for an explicit `additionalKeyPatterns: ["input_tokens"]` case through captureException/captureMessage.
Flagged by Autter security & observability checks.
There was a problem hiding this comment.
🔴 Autter review in progress — running security, correctness & dependency checks on this PR. Follow live step-by-step progress on the autter/review-gate check in the merge box. Merge is blocked until the gate completes; Autter approves automatically when the review comes back clean, and releases this hold with a neutral review when it finds non-blocking issues.
| const lowered = key.toLowerCase(); | ||
|
|
||
| // Only explicitly supported GenAI usage attributes bypass key redaction. | ||
| if (USAGE_TOKEN_KEYS.has(lowered)) return false; |
There was a problem hiding this comment.
🟠 [ai] Code correctness issue — Risk: 75/100
This unconditional exemption bypasses all configured redaction patterns for generic keys such as token_count, total_tokens, and input_tokens. An arbitrary attribute using one of these names can now expose a sensitive string or other secret value, and callers cannot explicitly redact it. This affects the exported redactAttributes path and makeSafeCapture in packages/runtime-node/src/server.ts. This touches exported/public surface code, so impact assessment should consider downstream callers. Blast radius — if this defect reaches production it can fail the downstream usage that depends on this file: functions redactAttributes, redactValue, isSensitiveKey, redactWith, makeRedactor, redactString; scopes @autter/runtime-node; dependent files @opentelemetry/api.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Functions/symbols:
redactAttributes,redactValue,isSensitiveKey,redactWith,makeRedactor,redactString - Dependent files:
@opentelemetry/api - Scopes:
@autter/runtime-node
🛠 AI fix prompt (copy & paste into your coding agent)
Preserve user-configured redaction for arbitrary attributes: exempt only canonical GenAI usage attributes when their values are valid non-negative numeric token counts, or otherwise apply the configured key redaction before bypassing. Add tests covering sensitive string values and custom redaction patterns on `token_count`/`input_tokens`, while retaining numeric GenAI usage counts. Blast radius — if this defect reaches production it can fail the downstream usage that depends on this file: functions `redactAttributes`, `redactValue`, `isSensitiveKey`, `redactWith`, `makeRedactor`, `redactString`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.
Flagged by Autter security & observability checks.
|
| const lowered = key.toLowerCase(); | ||
|
|
||
| // Only explicitly supported GenAI usage attributes bypass key redaction. | ||
| if (USAGE_TOKEN_KEYS.has(lowered)) return false; |
There was a problem hiding this comment.
🟠 [ai] Token-count allowlist bypasses guaranteed redaction and custom key patterns — Risk: 78/100
The new early return makes exact keys such as token_count and input_tokens bypass every compiled key pattern, including caller-supplied additionalKeyPatterns. This breaks the documented contract for makeSafeCapture, whose callers use it as a guaranteed-redaction boundary, and for RedactOptions, where matching additional key patterns are supposed to mask the whole value. A library calling safe.captureException(error, { token_count: secret }) or configuring redactAttributes({ additionalKeyPatterns: ['token_count'] }) now forwards the value unchanged instead of masking it.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Dependent files:
packages/runtime-node/src/redact.ts,packages/runtime-node/src/server.ts
🛠 AI fix prompt (copy & paste into your coding agent)
Apply the usage exception only to the built-in GenAI usage attributes, or evaluate caller-supplied additionalKeyPatterns before the usage allowlist. Preserve makeSafeCapture's guarantee that every sensitive/custom-matched attribute is redacted.
Flagged by Autter security & observability checks.
| const lowered = key.toLowerCase(); | ||
|
|
||
| // Only explicitly supported GenAI usage attributes bypass key redaction. | ||
| if (USAGE_TOKEN_KEYS.has(lowered)) return false; |
There was a problem hiding this comment.
🟠 [ai] Usage-key exemption can leak arbitrary sensitive values during error capture — Risk: 78/100
The exact-key bypass disables the whole-value sensitive-key mask for every value stored under an allowlisted usage name, but it does not validate that the value is a numeric token count. If an exception or failure path passes an attacker-controlled or accidentally mis-keyed string such as gen_ai.usage.input_tokens: "database password: ...", redactValue only applies the finite value-pattern scrubbers and then returns any text those patterns do not recognize. The previous key-level behavior would have replaced the entire value before the span was exported, so this change is not safe unless the exemption is restricted to validated numeric usage values (or the value is otherwise guaranteed not to contain secrets).
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Dependent files:
packages/runtime-node/src/redact.ts,packages/runtime-node/src/server.ts
🛠 AI fix prompt (copy & paste into your coding agent)
Keep the usage exemption only when the value is a finite numeric token count, or apply a type/value validation before bypassing key redaction; non-numeric or object/string values under usage keys should remain fully masked.
Flagged by Autter security & observability checks.
There was a problem hiding this comment.
🔴 Autter review in progress — running security, correctness & dependency checks on this PR. Follow live step-by-step progress on the autter/review-gate check in the merge box. Merge is blocked until the gate completes; Autter approves automatically when the review comes back clean, and releases this hold with a neutral review when it finds non-blocking issues.
|
There was a problem hiding this comment.
Autter completed PR review for #17: 4 finding(s) remain below the merge-blocking bar, so this review stays neutral rather than approving. See the findings below; the task checklist follows as the review's final comment.
|
|
||
| // Canonical GenAI usage attributes are safe when they contain | ||
| // valid non-negative numeric counts. | ||
| if (USAGE_TOKEN_KEYS.has(lowered)) { |
There was a problem hiding this comment.
🟠 [ai] Canonical usage exemption ignores additional key redaction patterns — Risk: 56/100
The canonical GenAI branch returns before checking r.keyPatterns, so a caller-supplied additionalKeyPatterns match can never redact a valid numeric value under a canonical usage key such as gen_ai.usage.input_tokens. This is reachable through the documented redactAttributes(attributes, { additionalKeyPatterns: [...] }) API and violates the option contract that extra key patterns extend the built-in key rules, allowing a configured policy to be silently bypassed for these attributes.
⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:
- Dependent files:
packages/runtime-node/src/redact.ts
🛠 AI fix prompt (copy & paste into your coding agent)
Evaluate the configured key patterns before applying the canonical GenAI numeric exemption, while retaining the exemption only when no configured pattern matches and the value is a finite non-negative number.
Flagged by Autter security & observability checks.
| const out = redactAttributes({ | ||
| "gen_ai.usage.input_tokens": "secret", | ||
| "gen_ai.usage.output_tokens": -1, | ||
| "gen_ai.usage.total_tokens": NaN, |
There was a problem hiding this comment.
🟠 [deterministic] Biome: lint/style/useNumberNamespace — Risk: 55/100
Use Number.NaN instead of the equivalent global.
🛠 AI fix prompt (copy & paste into your coding agent)
Fix the Biome `lint/style/useNumberNamespace` issue at packages/runtime-node/test/redact.test.mjs:158: Use Number.NaN instead of the equivalent global.
Flagged by Autter security & observability checks.
| "gen_ai.usage.input_tokens": "secret", | ||
| "gen_ai.usage.output_tokens": -1, | ||
| "gen_ai.usage.total_tokens": NaN, | ||
| "gen_ai.usage.token_count": Infinity, |
There was a problem hiding this comment.
🟠 [deterministic] Biome: lint/style/useNumberNamespace — Risk: 55/100
Use Number.POSITIVE_INFINITY instead of the equivalent global.
🛠 AI fix prompt (copy & paste into your coding agent)
Fix the Biome `lint/style/useNumberNamespace` issue at packages/runtime-node/test/redact.test.mjs:159: Use Number.POSITIVE_INFINITY instead of the equivalent global.
Flagged by Autter security & observability checks.
Autter task list
Generated from PR diff, blast radius, and context. Issues found
🛠 Fix optionsCheck one option and Autter will start a fix run for the unresolved issues above.
Checking a box triggers the fix run immediately — Autter comments back with the issues being fixed and the branch created for each. |
|
Hi Sagnik, |
2 participants
Problem
The server-side attribute redactor masks any key matching the sensitive-key patterns.
/token/is an unanchored substring, so besides real secrets it also masks non-secret usage attributes:gen_ai.usage.input_tokens,gen_ai.usage.output_tokens(OpenTelemetry GenAI semconv)prompt_tokens,completion_tokens,total_tokens,token_count,max_tokensFor an SDK that ships
llmTracing, silently turning token counts into[redacted]loses exactly the usage data teams want to see.It's also inconsistent with this file's own convention — the comment above the list notes patterns are "anchored where a loose substring would over-redact (
cardmust not eatdiscard,authormust not eatauthor_id)" — but/token/was left loose.Fix
Anchor the pattern to
/token$/. It still masks secret token keys (token,access_token,refresh_token,authToken, …) while leaving usage counts (*_tokens,token_count,max_tokens) intact. Value-level scrubbing (JWT / bearer / provider-key patterns) still redacts token-shaped values, so defense-in-depth is unchanged.Tests
Added two cases to
redact.test.mjs: one asserting usage token-counts pass through untouched, one asserting secret*tokenkeys are still masked.node --test→ all green.Scope
One-line behaviour change + tests. No API or wire-format change.
Summary
Summary generated by Autter.
Fixes server-side attribute redaction in
@autter/runtime-nodeso canonical GenAI and LLM usage token-count attributes remain available when their values are numeric, while sensitive token-like fields and non-numeric values continue to be masked.Changes
redactValueand sensitive-key handling inpackages/runtime-node/src/redact.tsto preserve approved numeric usage-count attributes without weakening redaction for secrets.redactAttributesbehavior remains unchanged for other sensitive data.Acceptance Criteria
redactAttributes.Test Plan
npm run buildfrom the repository root.discard_countandauthor_id.Rollback Plan
Revert commit
29a86db7bb2abdfb5f925d660a01ac8fe89d8014and redeploy@autter/runtime-node; this restores the previous redaction implementation without requiring data or schema rollback.Related Issues
No linked issue was identified.
Written for commit 29a86db. Summary will update on new commits.