Skip to content

feat: privacy-preserving redaction pipeline for operational sinks - #1235

Merged
Mosas2000 merged 1 commit into
StellaBridge:mainfrom
Majormaxx:feat/privacy-redaction-pipeline
Aug 30, 2026
Merged

feat: privacy-preserving redaction pipeline for operational sinks#1235
Mosas2000 merged 1 commit into
StellaBridge:mainfrom
Majormaxx:feat/privacy-redaction-pipeline

Conversation

@Majormaxx

@Majormaxx Majormaxx commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a schema-aware redaction pipeline that classifies sensitive operational fields centrally and enforces sink-specific redaction before anything is persisted or transmitted. Sensitive material (Stellar/EVM addresses, transaction hashes, IPs, notes/evidence, endpoint details, secrets) is pseudonymized or redacted deterministically before reaching audit, webhook, export (PDF/CSV/JSON), websocket, and log sinks.

What changed

  • New module backend/src/privacy/:
    • fieldRegistry.ts — central sensitivity classification with **/* path matching, rule fingerprinting, and a stable registry singleton.
    • secretScanner.ts — detects Stellar ed25519 secrets, EVM private keys, BIP-39 mnemonics, JWTs, bearer tokens, and named secrets.
    • pseudonymizer.ts — deterministic HMAC-SHA-256 pseudonyms (psn: prefixed), namespaced and irreversible.
    • redaction.service.ts — engine applying per-sink policies (audit, webhook, export, websocket, log, ...); optional block-on-secret for irreversible sinks; re-walks stringified JSON bodies.
    • redactionDecision.service.ts — best-effort, versioned decision recording (fingerprints and paths only, never original values).
  • Sink wiring (all before persistence/transmission):
    • Audit: audit.service.ts redacts then computes checksum.
    • Webhook: webhook.service.ts + webhook.service.outbox.ts redact payloads in both single and batch paths.
    • Export: export.worker.ts wraps the row stream through the export policy.
    • WebSocket: both websocket.ts (System A) and websocket.server.ts (System B) redact before delivery/Redis publish.
    • Logs: logger.ts derives pino redact paths from the central registry.
  • Config: REDACTION_ENABLED, pseudonym salt/namespace, block-on-secret, and decision-log flags.
  • Migration: redaction_decisions audit table for post-hoc decision review.
  • Tests: 31 unit tests in tests/services/privacyRedaction.service.test.ts.

Verification

  • tsc --noEmit clean for all touched files.
  • ESLint clean for all touched files.
  • vitest run tests/services/privacyRedaction.service.test.ts → 31 passing.

Notes

The webhook outbox path embeds endpoint.secret/endpoint.url into outbox_events (pre-existing behavior). Payload redaction is applied, but the endpoint secret itself is intentionally left unchanged to avoid breaking the event dispatcher contract.

Closes #1018

@Mosas2000

Copy link
Copy Markdown
Contributor

A perfect one as this centralized redaction pipeline with deterministic pseudonymization is an outstanding approach to data privacy and compliance, seamlessly integrating it across all egress sinks ensures secrets never leak while preserving full auditability. Thank you.

@Mosas2000
Mosas2000 merged commit 00d2019 into StellaBridge:main Aug 30, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

security: Build Privacy-Preserving Operational Data Redaction Pipeline

2 participants