Skip to content

Masked entities are no longer persisted to SQLite #464

@hanneshapke

Description

@hanneshapke

Summary

Masked entities are not being persisted in the SQLite database. This worked in earlier releases and has regressed — we need to figure out what broke and restore the persistence path.

Code path to review

PII mappings and log entries are stored by the SQLitePIIMappingDB in the backend:

  • src/backend/pii/database.go:185StoreMapping(ctx, original, dummy, piiType, confidence) upserts into the pii_mappings table (this is the entity → dummy mapping store)
  • src/backend/pii/database.go:293InsertLog(ctx, message, direction, entities, blocked) writes the per-request log row with the JSON-marshalled entities into the logs table
  • src/backend/pii/mapper.go:35 — the only non-test caller of StoreMapping; this is where the masking pipeline persists each detected entity

Callers in the request/response pipeline:

  • src/backend/proxy/handler.go:333InsertLog(..., \"request_original\", entities, ...)
  • src/backend/proxy/handler.go:339InsertLog(..., \"request_masked\", entities, ...)
  • src/backend/proxy/handler.go:365 / :371 — response-side InsertLog calls (note: these intentionally pass []pii.Entity{})

Suggested next steps

  1. Turn on debug mode (SetDebugMode(true)src/backend/pii/database.go:360) and confirm whether [InsertLog] and the StoreMapping path are actually being hit on a real request.
  2. Inspect cache.db after a request: SELECT count(*) FROM pii_mappings; and SELECT count(*), direction FROM logs GROUP BY direction; — this will tell you whether the regression is at the StoreMapping layer, the InsertLog layer, or upstream in mapper.go / handler.go.
  3. git log -p src/backend/pii/database.go src/backend/pii/mapper.go src/backend/proxy/handler.go to find the change that broke persistence.
  4. Add a regression test alongside src/backend/pii/database_test.go covering the end-to-end "request masked → mapping row exists → log row exists" flow so this doesn't silently regress again.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions