Conversation
…er matches analyze() assigned region_code_for_number()'s result back to the region it was iterating, so an international number that parsed cleanly overwrote the region used to explain every later match in that pass. Hold it per match instead.
|
Second verification pass on this same root cause from this account (two of my sessions worked it in 1. The empty-region case renders as It is reachable through the documented 2. 3. Evidence you can fold in, all re-measured against
Happy to hand over the test block or the corpus script if you want them in the Verification section rather |
Problem
PhoneRecognizer.analyzeassigned the region detected for one match back toregion, the variable the outer loop is iterating, so it leaked into every later match in that pass and produced ananalysis_explanationnaming a region that never matched the number.Reproduces with the default recognizer, no configuration — on
main@f251c513:Swapping the two numbers makes both labels correct, so the behaviour is order-dependent — which is also why the suite does not catch it: every multi-number case in
test_when_phone_with_textual_explanation_then_succeedputs the national-format number first.Mechanism:
phonenumbers.parse()is called without adefault_region, so national-format input always raisesNumberParseExceptionand lands in theexceptbranch, which reports whateverregioncurrently holds. After any international number in the same pass, that is the previous match's region.Full write-up with both orders: #2268
Change
Keep the detected value in a per-match name and never overwrite the loop variable; the
exceptbranch then reports the region the match was actually found under.try: parsed_number = phonenumbers.parse(text[match.start : match.end]) - region = phonenumbers.region_code_for_number(parsed_number) - results += [ - self._get_recognizer_result(match, text, region, nlp_artifacts) - ] + matched_region = phonenumbers.region_code_for_number(parsed_number) except NumberParseException: - results += [ - self._get_recognizer_result(match, text, region, nlp_artifacts) - ] + matched_region = region + results += [ + self._get_recognizer_result( + match, text, matched_region, nlp_artifacts + ) + ]The two identical
results += [...]blocks collapse into one as a consequence of the rename, which is why the diff is 7 lines rather than 2. Detection logic,PhoneNumberMatcherarguments, scores and dedup are untouched.Verification
One venv with the project installed and the spaCy model present, Python 3.11; each command repeated on the pristine
f251c513tree as a control (file hashes compared so the "base" run provably loaded base code).pytest tests/test_phone_recognizer.py tests/test_ph_mobile_number_recognizer.py tests/test_tr_phone_number_recognizer.py tests/test_recognizers_loader_utils.py tests/test_context_support.py -q1 failed, 436 passed, 2 skipped437 passed, 2 skippedGBthenUS. It fails on base with the exact mislabel from the issue, so it pins the behaviour rather than the implementation.+44first →GB,US;(415)first →US,GB. Spans and scores (0.4) are unchanged in every case, so the only thing the patch alters is the label text.ruff checkfrom the repository root with the version CI pins (requirements-ruff.txt→ ruff 0.9.2): All checks passed. CI's lint job runs onlyruff check(.github/workflows/ci.yml:38-39);ruff formatwould reformattests/test_phone_recognizer.pyonmainas well, so I left the file's existing layout alone rather than mix a formatter sweep into this change.presidio-analyzer/tests/and the e2e suite. A fullpytest testscollects modules that make real HuggingFace/Azure/stanza network calls in this environment, so I scoped to the files that exercise this recognizer plus the ones that import it, instead of reporting an unrun green.Note on tracking: #2268 is the report for this defect. A second issue (#2269) describing the same root cause was filed from this same account about a minute later while two of my sessions were working in parallel; it has been closed as a duplicate so the report lives in one place.
Scope and limits
region_code_for_numberreturns an empty string (a parsed number with no assigned region) — today that renders asRecognized as region phone number, and whether it should fall back to the iterating region is a separate product decision. Happy to fold it in if you want it.presidio-analyzer/tests/test_phone_recognizer.pytoo. Different concern — it is aboutDEFAULT_SUPPORTED_REGIONSusingGBrather thanUK— but whichever lands second needs a one-line rebase.CHANGELOG.mdentry added underAnalyzer → Fixed, following the precedent of fix(analyzer): use valid region code GB instead of UK in PhoneRecognizer #2174 in this same recognizer.Opening as draft:
docs/development.md:70asks for an issue before a PR and:73for two maintainer approvals, so I would rather this sit visibly until someone confirms the direction and the empty-region question above.Disclosure: prepared with an AI coding assistant. Every number above comes from a command run against
main@f251c513and repeated on the unpatched tree with the identical command line; the recognizer output blocks were each produced by running that exact text.