Skip to content

feat: auto-flush on process exit + PII redaction across all packages - #16

Merged
sagnik11 merged 2 commits into
mainfrom
feat/auto-flush-and-redaction
Aug 25, 2026
Merged

feat: auto-flush on process exit + PII redaction across all packages#16
sagnik11 merged 2 commits into
mainfrom
feat/auto-flush-and-redaction

Conversation

@sagnik11

Copy link
Copy Markdown
Member

Problem

Two real-world DX failures found while testing the SDK end-to-end:

  1. Silent data loss when shutdown() is forgotten. With default batching (~2 s errors, ~5 s traces, 60 s metrics), exiting without awaiting autter.shutdown() dropped everything still buffered — no warning, nothing.
  2. The "no emails" privacy line didn't cover custom attributes. captureException(err, { "user.email": … }) was forwarded verbatim on the server side, and browser context bags were stored verbatim ingest-side — while the browser relay whitelist-sanitises its whole payload.

Changes

@autter/runtime-node

  • installAutterAutoFlush() (installed by default via initAutterServer({ autoFlush })) — force-flushes every exporter on beforeExit/SIGINT/SIGTERM:
    • coexists with the app's own signal handlers (flushes alongside, never changes their exit path; conventional 130/143 codes only when we're the sole handler)
    • timeout-bounded (default 3 s); a second signal always exits immediately
    • flushes via the span processors we handed NodeSDK directly (NodeSDK exposes no forceFlush())
  • redactAttributes() / makeSafeCapture() (on by default, { redactAttributes: false } opts out) — masks emails, JWTs, sk-/ghp_/AWS/Slack tokens, bearer headers, scheme://user:pass@ URLs, and sensitive-keyed attributes (password, token, secret, cookie, …) before export; covers captureException, captureMessage, withProcessSpan, and LLM attributes
  • Debug mode (debug: true or AUTTER_DEBUG=1) — [autter] exported N span(s) lines plus an explicit stderr warning if the process exits with captures never confirmed exported

Other packages

  • @autter/runtime-browser — new redactContext(): same scrubbing applied to custom context at the enqueue() choke point before anything leaves the page. Bundle stays at ~1.3 kB under the 5 kB size-limit.
  • @autter/otlp-ingesterevent.context is scrubbed at normalize time before ClickHouse: defense-in-depth that also protects payloads from outdated SDK versions.
  • @autter/runtime-next — re-exports makeSafeCapture / installAutterAutoFlush / redactAttributes; registerAutter already passes the new options through.

Drive-by fix

  • runtime-browser's build never emitted dist/index.d.ts despite declaring it — runtime-next's DTS build only worked off a stale artifact; now deterministic (--dts).

Tests

44 tests across three zero-dependency node:test suites (no new deps):

  • node (17): redaction unit coverage incl. anti-overmatch cases (discard_count, author_id, card_brand), plus child-process e2e: SIGTERM sole-handler → flush+exit 143, app-owned signal → app's exit code wins, double-signal cuts through, beforeExit flush completes without looping, and a wire-level e2e asserting no raw email/JWT/bearer reaches a local collector while [redacted] does
  • browser (5) + ingester (4 new / 22 total): source- and storage-side scrubbing

Test fixtures assemble fake tokens from string fragments so GitHub push protection doesn't mistake them for real credentials. CI now runs all three suites.

…stom attributes

- installAutterAutoFlush(): force-flushes all exporters on beforeExit/SIGINT/
  SIGTERM so a forgotten shutdown() no longer silently drops buffered telemetry.
  Installed by default via initAutterServer({ autoFlush }); coexists with the
  app's own signal handlers and never changes their exit path. Timeout-bounded,
  second signal exits immediately, conventional 130/143 exit codes.
- redactAttributes()/makeSafeCapture(): masks emails, JWTs, sk-/ghp_/AWS/Slack
  tokens, bearer headers, scheme://user:pass@ URLs and sensitive-keyed
  attributes before they leave the process — closing the server-side gap to
  match the browser relay's payload whitelist. On by default
  ({ redactAttributes: false } opts out); covers captureException,
  captureMessage, withProcessSpan and LLM attributes.
- debug mode (debug: true or AUTTER_DEBUG=1): 'exported N span(s)' lines plus a
  stderr warning when the process exits with unconfirmed captures.
- node:test suite (17 tests) incl. an e2e asserting redaction on the wire;
  wired into CI. Test tokens are fragment-assembled so secret scanners don't
  mistake fixtures for real credentials.
…orage, Next.js re-exports

- runtime-browser: new redactContext() masks sensitive-keyed values and
  email-shaped strings in custom context before anything leaves the page,
  applied at the enqueue choke point (beforeSend sees the final form).
  Bundle stays at ~1.3 kB under the 5 kB size-limit. The 'never sent:
  emails' claim is now actually true for custom context.
- otlp-ingester: scrub event.context at normalize time before it reaches
  ClickHouse — defense-in-depth that also protects payloads from outdated
  SDK versions.
- runtime-next: re-export makeSafeCapture / installAutterAutoFlush /
  redactAttributes (+ types); registerAutter already passes the new
  initAutterServer options through. README documents the defaults.
- fix(runtime-browser): build script now emits dist/index.d.ts (--dts) to
  match its declared types entry — @autter/runtime-next's DTS build
  depended on it and broke on a clean checkout once --clean wiped the
  stale artifact.
- tests: browser redaction suite (5) + ingester normalize-browser suite (4);
  CI runs both alongside the node suite.
@sagnik11
sagnik11 merged commit 49ecfad into main Aug 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant