Summary
Masked entities are not being persisted in the SQLite database. This worked in earlier releases and has regressed — we need to figure out what broke and restore the persistence path.
Code path to review
PII mappings and log entries are stored by the SQLitePIIMappingDB in the backend:
src/backend/pii/database.go:185 — StoreMapping(ctx, original, dummy, piiType, confidence) upserts into the pii_mappings table (this is the entity → dummy mapping store)
src/backend/pii/database.go:293 — InsertLog(ctx, message, direction, entities, blocked) writes the per-request log row with the JSON-marshalled entities into the logs table
src/backend/pii/mapper.go:35 — the only non-test caller of StoreMapping; this is where the masking pipeline persists each detected entity
Callers in the request/response pipeline:
src/backend/proxy/handler.go:333 — InsertLog(..., \"request_original\", entities, ...)
src/backend/proxy/handler.go:339 — InsertLog(..., \"request_masked\", entities, ...)
src/backend/proxy/handler.go:365 / :371 — response-side InsertLog calls (note: these intentionally pass []pii.Entity{})
Suggested next steps
- Turn on debug mode (
SetDebugMode(true) — src/backend/pii/database.go:360) and confirm whether [InsertLog] and the StoreMapping path are actually being hit on a real request.
- Inspect
cache.db after a request: SELECT count(*) FROM pii_mappings; and SELECT count(*), direction FROM logs GROUP BY direction; — this will tell you whether the regression is at the StoreMapping layer, the InsertLog layer, or upstream in mapper.go / handler.go.
git log -p src/backend/pii/database.go src/backend/pii/mapper.go src/backend/proxy/handler.go to find the change that broke persistence.
- Add a regression test alongside
src/backend/pii/database_test.go covering the end-to-end "request masked → mapping row exists → log row exists" flow so this doesn't silently regress again.
Summary
Masked entities are not being persisted in the SQLite database. This worked in earlier releases and has regressed — we need to figure out what broke and restore the persistence path.
Code path to review
PII mappings and log entries are stored by the
SQLitePIIMappingDBin the backend:src/backend/pii/database.go:185—StoreMapping(ctx, original, dummy, piiType, confidence)upserts into thepii_mappingstable (this is the entity → dummy mapping store)src/backend/pii/database.go:293—InsertLog(ctx, message, direction, entities, blocked)writes the per-request log row with the JSON-marshalled entities into thelogstablesrc/backend/pii/mapper.go:35— the only non-test caller ofStoreMapping; this is where the masking pipeline persists each detected entityCallers in the request/response pipeline:
src/backend/proxy/handler.go:333—InsertLog(..., \"request_original\", entities, ...)src/backend/proxy/handler.go:339—InsertLog(..., \"request_masked\", entities, ...)src/backend/proxy/handler.go:365/:371— response-sideInsertLogcalls (note: these intentionally pass[]pii.Entity{})Suggested next steps
SetDebugMode(true)—src/backend/pii/database.go:360) and confirm whether[InsertLog]and theStoreMappingpath are actually being hit on a real request.cache.dbafter a request:SELECT count(*) FROM pii_mappings;andSELECT count(*), direction FROM logs GROUP BY direction;— this will tell you whether the regression is at theStoreMappinglayer, theInsertLoglayer, or upstream inmapper.go/handler.go.git log -p src/backend/pii/database.go src/backend/pii/mapper.go src/backend/proxy/handler.goto find the change that broke persistence.src/backend/pii/database_test.gocovering the end-to-end "request masked → mapping row exists → log row exists" flow so this doesn't silently regress again.