feat(pii): add UK NINO detection and redaction to Tier 1 - #11
feat(pii): add UK NINO detection and redaction to Tier 1#11snowyukitty wants to merge 2 commits into
Conversation
|
Merged current
Verification at the merge head
Tier 1 overhead, since CONTRIBUTING.md sets a sub-millisecond bar for compiled Tier 1 regex. Measured over 2000 rounds on an 800-character payload containing an email, a phone number, an SSN, a card number, an IP and a NINO, Tier 1 only, comparing
Separately, timing Synthetic-mode placeholder for In synthetic mode,
Which of those you want is a call about your product rather than about the pattern, so I have left it to you. |
Description
This PR expands Tier 1 PII detection and redaction coverage for UK National Insurance Numbers (NINO) in
llm_shield_proxy/engines/pii_engine.py.Changes
UK_NINOregex toTIER1_PATTERNSaimed at the usual HMRC letter/digit layout:[A-CEGHJ-PR-TW-Z](excluding HMRC disallowed charactersD,F,I,Q,U,V)[A-CEGHJ-NPR-TW-Z](excludingD,F,I,Q,U,V,O)GB,BG,NK,KN,TN,NT,ZZ) case-insensitively[A-D][UK_NINO_X]through the existing entity-type naming. This PR does not add a shape-preserving synthetic token forUK_NINO.tests/test_pii_engine.pyfor positive NINO samples, the seven excluded prefix pairs, forbidden prefix characters, invalid suffixes, and invalid lengths.Against current
mainthe branch is 2 files, +76/-0.Testing
Run at head
97434e0, after merging currentmain:pytest tests/test_pii_engine.py— 12 passed, including both NINO tests.ruff check .— clean.bandit -r llm_shield_proxy/ -s B101,B104,B311— clean.pytest --ignore=tests/k8s --ignore=tests/k8s_ootb --ignore=tests/ootb— 110 passed, 1 failed. The failure,tests/test_grpc_ext_proc.py::test_ext_proc_redacts_request_body, reproduces on a detached checkout ofmainwith this branch absent; it is not introduced here.Tier 1 latency
Per CONTRIBUTING.md's sub-millisecond bar for compiled Tier 1 regex. 2000 rounds, 800-character payload, Tier 1 only, with and without the one added pattern: +0.023 ms median, +0.020 ms at p95. Separately,
pattern.search()for the added regex alone on five adversarial inputs up to 80k characters each completed in under 0.9 ms on the same machine.Closes #7