Skip to content

fix(analyzer): add Korean context terms to KrRrnRecognizer (#2212) - #2258

Open
bodapatisaikrishna wants to merge 1 commit into
data-privacy-stack:mainfrom
bodapatisaikrishna:fix/kr-rrn-korean-context
Open

bodapatisaikrishna wants to merge 1 commit into
data-privacy-stack:mainfrom
bodapatisaikrishna:fix/kr-rrn-korean-context

Conversation

@bodapatisaikrishna

Copy link
Copy Markdown

Change Description

Fixes an omission in KrRrnRecognizer.CONTEXT where only English terms ("Korean RRN", "RRN", etc.) were defined, unlike all other Korean recognizers (KrDriverLicenseRecognizer, KrPassportRecognizer, KrFrnRecognizer, KrBrnRecognizer) which include native Korean keywords.

Because Korean Resident Registration Numbers (주민등록번호) primarily appear in Korean documents and text, context enhancement was never firing for native Korean context phrases. This was especially problematic for post-October 2020 RRNs (which lack the pre-2020 checksum algorithm and return pattern score 0.5), leaving them below typical analyzer acceptance thresholds without the context boost.

Changes:

  • Added standard Korean context terms to KrRrnRecognizer.CONTEXT:
    • "주민등록번호" (Resident Registration Number)
    • "주민번호" (short form: Resident Number)
    • "주민등록증" (Resident Registration Card / ID card)
    • "주민등록" (Resident Registration)
    • "신분증" (ID card / Identification)
    • "본인인증" (Identity Verification)
  • Added unit tests in presidio-analyzer/tests/test_kr_rrn_recognizer.py:
    • Validated presence of Korean context keywords in KrRrnRecognizer.context.
    • Tested end-to-end score enhancement using LemmaContextAwareEnhancer (score boosted from 0.5 to 0.85 with supportive_context_word set).
    • Added negative control test ensuring unrelated context does not boost scores.
    • Added Korean context sentences to parametrized recognition tests.

Issue reference

Fixes #2212

Checklist

  • I have reviewed the contribution guidelines
  • I agree to follow this project's Code of Conduct
  • I confirm that I have the right to submit this contribution and that it does not knowingly contain proprietary or confidential code.
  • My code includes unit tests
  • All unit tests and lint checks pass locally
  • My PR contains documentation updates / additions if required

…acy-stack#2212)

KrRrnRecognizer.CONTEXT only contained English keywords ('Korean RRN', 'RRN', etc.), so context enhancement never fired for Korean-language documents where RRNs appear. All other Korean recognizers already define native Korean context terms.

Adds '주민등록번호', '주민번호', '주민등록증', '주민등록', '신분증', and '본인인증' to KrRrnRecognizer.CONTEXT and adds unit tests covering context presence and score enhancement.

Fixes data-privacy-stack#2212.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KR_RRN context enhancement never fires on Korean-language text

1 participant