Skip to content

feat(pii): add UK NINO detection and redaction to Tier 1 - #11

Open
snowyukitty wants to merge 2 commits into
ninadphalak:mainfrom
snowyukitty:feat/uk-nino-pii-detection
Open

feat(pii): add UK NINO detection and redaction to Tier 1#11
snowyukitty wants to merge 2 commits into
ninadphalak:mainfrom
snowyukitty:feat/uk-nino-pii-detection

Conversation

@snowyukitty

@snowyukitty snowyukitty commented Aug 16, 2026

Copy link
Copy Markdown

Description

This PR expands Tier 1 PII detection and redaction coverage for UK National Insurance Numbers (NINO) in llm_shield_proxy/engines/pii_engine.py.

Changes

  • Added a compiled UK_NINO regex to TIER1_PATTERNS aimed at the usual HMRC letter/digit layout:
    • 1st prefix character: [A-CEGHJ-PR-TW-Z] (excluding HMRC disallowed characters D, F, I, Q, U, V)
    • 2nd prefix character: [A-CEGHJ-NPR-TW-Z] (excluding D, F, I, Q, U, V, O)
    • Negative lookahead for all 7 excluded HMRC pairs (GB, BG, NK, KN, TN, NT, ZZ) case-insensitively
    • Supports standard space, hyphen, and contiguous formats with valid suffix letters [A-D]
  • In non-synthetic vault mode, matches are labelled [UK_NINO_X] through the existing entity-type naming. This PR does not add a shape-preserving synthetic token for UK_NINO.
  • Added tests in tests/test_pii_engine.py for positive NINO samples, the seven excluded prefix pairs, forbidden prefix characters, invalid suffixes, and invalid lengths.

Against current main the branch is 2 files, +76/-0.

Testing

Run at head 97434e0, after merging current main:

  • pytest tests/test_pii_engine.py — 12 passed, including both NINO tests.
  • ruff check . — clean. bandit -r llm_shield_proxy/ -s B101,B104,B311 — clean.
  • pytest --ignore=tests/k8s --ignore=tests/k8s_ootb --ignore=tests/ootb — 110 passed, 1 failed. The failure, tests/test_grpc_ext_proc.py::test_ext_proc_redacts_request_body, reproduces on a detached checkout of main with this branch absent; it is not introduced here.

Tier 1 latency

Per CONTRIBUTING.md's sub-millisecond bar for compiled Tier 1 regex. 2000 rounds, 800-character payload, Tier 1 only, with and without the one added pattern: +0.023 ms median, +0.020 ms at p95. Separately, pattern.search() for the added regex alone on five adversarial inputs up to 80k characters each completed in under 0.9 ms on the same machine.

Closes #7

@snowyukitty

Copy link
Copy Markdown
Author

Merged current main into this branch, so it is no longer in conflict. Head is 97434e0.

main moved llm_shield_proxy/pii_engine.py to llm_shield_proxy/engines/pii_engine.py. Git carried the change across the rename; against current main this branch is still the same 2 files, +76/-0 — the UK_NINO entry in TIER1_PATTERNS and the two tests. The 76 added lines are byte-identical to the ones you already have (sha256 cd2b472c… over the added lines of both diffs), and nothing is removed.

Verification at the merge head

  • pytest tests/test_pii_engine.py — 12 passed, including both NINO tests.
  • ruff check . — clean. bandit -r llm_shield_proxy/ -s B101,B104,B311 — clean.
  • pytest --ignore=tests/k8s --ignore=tests/k8s_ootb --ignore=tests/ootb, the selection CI runs — 110 passed, 1 failed. The failure is tests/test_grpc_ext_proc.py::test_ext_proc_redacts_request_body (TypeError: must be called with a dataclass type or instance). It reproduces on a detached checkout of main at a48cf32 with this branch absent, and CI on main is red for a48cf32, cb05151 and b09777a. I have not touched it — flagging it only so the red check here is not read as this branch's.

Tier 1 overhead, since CONTRIBUTING.md sets a sub-millisecond bar for compiled Tier 1 regex. Measured over 2000 rounds on an 800-character payload containing an email, a phone number, an SSN, a card number, an IP and a NINO, Tier 1 only, comparing TIER1_PATTERNS with and without the one added entry:

without with delta
median 0.1507 ms 0.1735 ms +0.023 ms
p95 0.2152 ms 0.2355 ms +0.020 ms

Separately, timing pattern.search() for the added NINO regex alone on five adversarial inputs up to 80k characters (long partial prefixes, near-miss tails, 20k repeated separators, letters only, digits only): each completed in under 0.9 ms on the same machine.

Synthetic-mode placeholder for UK_NINO is not in this PR. I can follow up with whichever you prefer.

In synthetic mode, Vault.get_or_create_token maps entity types to shape-preserving fakes — EMAIL to a fake email, SSN to a fake SSN, CREDIT_CARD to a fake card. UK_NINO matches none of those branches and falls through to else: token = fake.word(), so a NINO is replaced by an ordinary word. Redaction and rehydration still work; non-synthetic mode gives [UK_NINO_1] like every other type, which is what the tests cover. The options I can see:

  1. Faker('en_GB').ssn(). On Faker 40.37.0, seeded, 200 calls produced values of the form ZZ604876T, ZZ759382T, ZZ219489T — every one prefixed ZZ and suffixed T — and 0 of the 200 matched the pattern in this PR. So the placeholder would not be re-detected by the engine, and vault.py currently holds one shared default-locale Faker().
  2. Generating a format-valid NINO preserves the shape, but manufactures strings that may match a real person's identifier.
  3. Leaving the generic-word fallback as it is.

Which of those you want is a call about your product rather than about the pattern, so I have left it to you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[i18n] Add regex pattern and validation for UK National Insurance Numbers (NINO)

1 participant