Skip to content

fix(runtime-node): stop redaction masking GenAI/usage token counts #17

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sagnik11 merged 4 commits into from
Aug 29, 2026
Merged

fix(runtime-node): stop redaction masking GenAI/usage token counts #17

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f31c9a3
fix(runtime-node): stop redaction masking GenAI/usage token counts
rajeshaipython-stack Aug 26, 2026
56c2227
fix(runtime-node): preserve usage token attributes
rajeshaipython-stack Aug 26, 2026
d001c3b
fix(runtime-node): tighten token usage redaction
rajeshaipython-stack Aug 26, 2026
29a86db
fix(runtime-node): tighten GenAI usage redaction
rajeshaipython-stack Aug 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 30 additions & 5 deletions packages/runtime-node/src/redact.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,16 +142,41 @@
if (typeof value === "object" && value !== null && depth > 0) {
const out: Record<string, unknown> = {};
for (const [k, v] of Object.entries(value as Record<string, unknown>)) {
out[k] = isSensitiveKey(k, r) ? r.mask : redactValue(v, r, depth - 1);
out[k] = isSensitiveKey(k, v, r) ? r.mask : redactValue(v, r, depth - 1);

Check warning on line 145 in packages/runtime-node/src/redact.ts

View check run for this annotation

Autter.dev / autter/review-gate

🟠 Medium · Missing CODEOWNERS reviewer approval 

packages/runtime-node/src/redact.ts has no configured CODEOWNERS entry in the supplied metadata and has no approving review; only comments or change requests are present. The affected redactAttributes, isSensitiveKey, and makeSafeCapture path can alter redaction of runtime telemetry. Blast radius — skipping this guardrail cascades to the downstream usage that depends on this file: functions `redactAttributes`, `redactValue`, `isSensitiveKey`, `redactWith`, `makeRedactor`, `redactString`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.
Suggested fix:
Configure or identify the CODEOWNERS owner for packages/runtime-node/src/redact.ts, then obtain an approving review from a matching owner before merging. Blast radius — skipping this guardrail cascades to the downstream usage that depends on this file: functions `redactAttributes`, `redactValue`, `isSensitiveKey`, `redactWith`, `makeRedactor`, `redactString`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.

Check warning on line 145 in packages/runtime-node/src/redact.ts

View check run for this annotation

Autter.dev / autter/review-gate

🟠 Medium · Missing CODEOWNERS reviewer approval 

packages/runtime-node/src/redact.ts has no configured CODEOWNERS entry in the supplied metadata and has no approving review; only comments or change requests are present. The affected redactAttributes, isSensitiveKey, and makeSafeCapture path can alter redaction of runtime telemetry. Blast radius — skipping this guardrail cascades to the downstream usage that depends on this file: functions `redactAttributes`, `redactValue`, `isSensitiveKey`, `redactWith`, `makeRedactor`, `redactString`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.
Suggested fix:
Configure or identify the CODEOWNERS owner for packages/runtime-node/src/redact.ts, then obtain an approving review from a matching owner before merging. Blast radius — skipping this guardrail cascades to the downstream usage that depends on this file: functions `redactAttributes`, `redactValue`, `isSensitiveKey`, `redactWith`, `makeRedactor`, `redactString`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.
}
return out;
}
return value;
}

function isSensitiveKey(key: string, r: CompiledRedactor): boolean {
const lowered = key.toLowerCase();
return r.keyPatterns.some((re) => re.test(lowered));
const USAGE_TOKEN_KEYS = new Set([

Check notice on line 152 in packages/runtime-node/src/redact.ts

View check run for this annotation

Autter.dev / autter/review-gate

🟡 Low · Missing linked tracker issue 

The PR description explicitly states that no linked issue was identified. This changes exported redactAttributes and its makeSafeCapture caller, affecting whether runtime telemetry token-count attributes reach downstream ingestion and storage. Blast radius — skipping this guardrail cascades to the downstream usage that depends on this file: functions `redactAttributes`, `redactValue`, `isSensitiveKey`, `redactWith`, `makeRedactor`, `redactString`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.
Suggested fix:
Add a valid tracker reference to the PR description, such as GitHub #123 or a Jira/Linear KEY-123, and explain how it tracks this redaction behavior change. Blast radius — skipping this guardrail cascades to the downstream usage that depends on this file: functions `redactAttributes`, `redactValue`, `isSensitiveKey`, `redactWith`, `makeRedactor`, `redactString`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.

Check notice on line 152 in packages/runtime-node/src/redact.ts

View check run for this annotation

Autter.dev / autter/review-gate

🟡 Low · Missing linked tracker issue 

The PR description explicitly states that no linked issue was identified. This changes exported redactAttributes and its makeSafeCapture caller, affecting whether runtime telemetry token-count attributes reach downstream ingestion and storage. Blast radius — skipping this guardrail cascades to the downstream usage that depends on this file: functions `redactAttributes`, `redactValue`, `isSensitiveKey`, `redactWith`, `makeRedactor`, `redactString`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.
Suggested fix:
Add a valid tracker reference to the PR description, such as GitHub #123 or a Jira/Linear KEY-123, and explain how it tracks this redaction behavior change. Blast radius — skipping this guardrail cascades to the downstream usage that depends on this file: functions `redactAttributes`, `redactValue`, `isSensitiveKey`, `redactWith`, `makeRedactor`, `redactString`; scopes `@autter/runtime-node`; dependent files `@opentelemetry/api`.
"gen_ai.usage.input_tokens",
"gen_ai.usage.output_tokens",
"gen_ai.usage.prompt_tokens",
"gen_ai.usage.completion_tokens",
"gen_ai.usage.total_tokens",
"gen_ai.usage.token_count",
]);

function isSensitiveKey(
key: string,
value: unknown,
r: CompiledRedactor,
): boolean {
const lowered = key.toLowerCase();

// Canonical GenAI usage attributes are safe when they contain
// valid non-negative numeric counts.
if (USAGE_TOKEN_KEYS.has(lowered)) {

Check warning on line 170 in packages/runtime-node/src/redact.ts

View check run for this annotation

Autter.dev / autter/review-gate

🟠 Medium · Canonical usage exemption ignores additional key redaction patterns 

The canonical GenAI branch returns before checking `r.keyPatterns`, so a caller-supplied `additionalKeyPatterns` match can never redact a valid numeric value under a canonical usage key such as `gen_ai.usage.input_tokens`. This is reachable through the documented `redactAttributes(attributes, { additionalKeyPatterns: [...] })` API and violates the option contract that extra key patterns extend the built-in key rules, allowing a configured policy to be silently bypassed for these attributes.
Suggested fix:
Evaluate the configured key patterns before applying the canonical GenAI numeric exemption, while retaining the exemption only when no configured pattern matches and the value is a finite non-negative number.

@autter-dev autter-dev Bot Aug 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 [ai] Canonical usage exemption ignores additional key redaction patterns — Risk: 56/100

The canonical GenAI branch returns before checking r.keyPatterns, so a caller-supplied additionalKeyPatterns match can never redact a valid numeric value under a canonical usage key such as gen_ai.usage.input_tokens. This is reachable through the documented redactAttributes(attributes, { additionalKeyPatterns: [...] }) API and violates the option contract that extra key patterns extend the built-in key rules, allowing a configured policy to be silently bypassed for these attributes.

⚠ Downstream affected — if this fails, it cascades to the usage that depends on this file:

  • Dependent files: packages/runtime-node/src/redact.ts
🛠 AI fix prompt (copy & paste into your coding agent) 
Evaluate the configured key patterns before applying the canonical GenAI numeric exemption, while retaining the exemption only when no configured pattern matches and the value is a finite non-negative number.

Flagged by Autter security & observability checks.

return !(
typeof value === "number" &&
Number.isFinite(value) &&
value >= 0
);
}

// All other sensitive keys, including token-like keys, are redacted.
return r.keyPatterns.some((re) => re.test(lowered));
}

/**
Expand All @@ -176,7 +201,7 @@
if (!attributes) return out;
for (const [key, value] of Object.entries(attributes)) {
if (value === undefined) continue;
out[key] = isSensitiveKey(key, r)
out[key] = isSensitiveKey(key, value, r)
? r.mask
: (redactValue(value, r, maxDepth) as Attributes[string]);
}
Expand Down
55 changes: 55 additions & 0 deletions packages/runtime-node/test/redact.test.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,3 +124,58 @@
assert.deepEqual(redactAttributes(), {});
assert.deepEqual(redactAttributes(null), {});
});

Check warning on line 127 in packages/runtime-node/test/redact.test.mjs

View check run for this annotation

Autter.dev / autter/review-gate

🟠 Medium · Missing linked tracker issue 

This PR's title and body do not reference any tracker issue (GitHub `#123`, Jira/Linear `KEY-123`, or `Fixes/Closes/Resolves`).
Suggested fix:
In the PR description, add a reference to the tracker issue this change implements (GitHub `#123`, Jira/Linear `PROJ-456`, or a `Fixes/Closes/Resolves` marker). Reviewers anchor on `packages/runtime-node/test/redact.test.mjs` around line 127 need that context to understand why this change exists and what success looks like. Why it matters: reviewers and on-call engineers need the linked issue to understand the why behind a change months from now.

Check warning on line 127 in packages/runtime-node/test/redact.test.mjs

View check run for this annotation

Autter.dev / autter/review-gate

🟠 Medium · Missing linked tracker issue 

This PR's title and body do not reference any tracker issue (GitHub `#123`, Jira/Linear `KEY-123`, or `Fixes/Closes/Resolves`).
Suggested fix:
In the PR description, add a reference to the tracker issue this change implements (GitHub `#123`, Jira/Linear `PROJ-456`, or a `Fixes/Closes/Resolves` marker). Reviewers anchor on `packages/runtime-node/test/redact.test.mjs` around line 127 need that context to understand why this change exists and what success looks like. Why it matters: reviewers and on-call engineers need the linked issue to understand the why behind a change months from now.
test("keeps canonical GenAI usage counts only when numeric", () => {

Check warning on line 128 in packages/runtime-node/test/redact.test.mjs

View check run for this annotation

Autter.dev / autter/review-gate

🟠 Medium · Missing CODEOWNERS reviewer approval 

The changed redaction regression tests have no configured CODEOWNERS entry in the supplied metadata and no approving review from a matching owner. These tests govern redactAttributes behavior used by runtime-node telemetry capture. Blast radius — skipping this guardrail cascades to the downstream usage that depends on this file: scopes `@autter/runtime-node`; dependent files `../dist/index.js`, `node:assert/strict`, `node:test`.
Suggested fix:
Configure or identify the CODEOWNERS owner for packages/runtime-node/test/redact.test.mjs, then obtain an approving review from a matching owner before merging. Blast radius — skipping this guardrail cascades to the downstream usage that depends on this file: scopes `@autter/runtime-node`; dependent files `../dist/index.js`, `node:assert/strict`, `node:test`.

Check warning on line 128 in packages/runtime-node/test/redact.test.mjs

View check run for this annotation

Autter.dev / autter/review-gate

🟠 Medium · Missing CODEOWNERS reviewer approval 

The changed redaction regression tests have no configured CODEOWNERS entry in the supplied metadata and no approving review from a matching owner. These tests govern redactAttributes behavior used by runtime-node telemetry capture. Blast radius — skipping this guardrail cascades to the downstream usage that depends on this file: scopes `@autter/runtime-node`; dependent files `../dist/index.js`, `node:assert/strict`, `node:test`.
Suggested fix:
Configure or identify the CODEOWNERS owner for packages/runtime-node/test/redact.test.mjs, then obtain an approving review from a matching owner before merging. Blast radius — skipping this guardrail cascades to the downstream usage that depends on this file: scopes `@autter/runtime-node`; dependent files `../dist/index.js`, `node:assert/strict`, `node:test`.
const out = redactAttributes({
"gen_ai.usage.input_tokens": 512,
"gen_ai.usage.output_tokens": 128,
"gen_ai.usage.prompt_tokens": 256,
"gen_ai.usage.completion_tokens": 128,
"gen_ai.usage.total_tokens": 640,
"gen_ai.usage.token_count": 42,
"gen_ai.usage.input_tokens_string": "secret",
max_tokens: 1000,
"secret.input_tokens": 999,
});

assert.deepEqual(out, {
"gen_ai.usage.input_tokens": 512,
"gen_ai.usage.output_tokens": 128,
"gen_ai.usage.prompt_tokens": 256,
"gen_ai.usage.completion_tokens": 128,
"gen_ai.usage.total_tokens": 640,
"gen_ai.usage.token_count": 42,
"gen_ai.usage.input_tokens_string": MASK,
max_tokens: MASK,
"secret.input_tokens": MASK,
});
});

test("redacts non-numeric canonical GenAI usage values", () => {
const out = redactAttributes({
"gen_ai.usage.input_tokens": "secret",
"gen_ai.usage.output_tokens": -1,
"gen_ai.usage.total_tokens": NaN,

Check warning on line 158 in packages/runtime-node/test/redact.test.mjs

View check run for this annotation

Autter.dev / autter/review-gate

🟠 Medium · Biome: lint/style/useNumberNamespace 

Use Number.NaN instead of the equivalent global.
Suggested fix:
Fix the Biome `lint/style/useNumberNamespace` issue at packages/runtime-node/test/redact.test.mjs:158: Use Number.NaN instead of the equivalent global.

@autter-dev autter-dev Bot Aug 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 [deterministic] Biome: lint/style/useNumberNamespace — Risk: 55/100

Use Number.NaN instead of the equivalent global.

🛠 AI fix prompt (copy & paste into your coding agent) 
Fix the Biome `lint/style/useNumberNamespace` issue at packages/runtime-node/test/redact.test.mjs:158: Use Number.NaN instead of the equivalent global.

Flagged by Autter security & observability checks.

"gen_ai.usage.token_count": Infinity,

Check warning on line 159 in packages/runtime-node/test/redact.test.mjs

View check run for this annotation

Autter.dev / autter/review-gate

🟠 Medium · Biome: lint/style/useNumberNamespace 

Use Number.POSITIVE_INFINITY instead of the equivalent global.
Suggested fix:
Fix the Biome `lint/style/useNumberNamespace` issue at packages/runtime-node/test/redact.test.mjs:159: Use Number.POSITIVE_INFINITY instead of the equivalent global.

@autter-dev autter-dev Bot Aug 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 [deterministic] Biome: lint/style/useNumberNamespace — Risk: 55/100

Use Number.POSITIVE_INFINITY instead of the equivalent global.

🛠 AI fix prompt (copy & paste into your coding agent) 
Fix the Biome `lint/style/useNumberNamespace` issue at packages/runtime-node/test/redact.test.mjs:159: Use Number.POSITIVE_INFINITY instead of the equivalent global.

Flagged by Autter security & observability checks.

});

assert.deepEqual(out, {
"gen_ai.usage.input_tokens": MASK,
"gen_ai.usage.output_tokens": MASK,
"gen_ai.usage.total_tokens": MASK,
"gen_ai.usage.token_count": MASK,
});
});
test("still masks secret token keys ending in token", () => {
const out = redactAttributes({
token: "raw",
access_token: "raw",
refresh_token: "raw",
authToken: "raw",
token_value: "raw",
tokenString: "raw",
token_id: "raw",
id_token_hint: "raw",
});
for (const value of Object.values(out)) assert.equal(value, MASK);
});
Loading