/analyze silently drops an unsupported entity when other requested entities are supported; IT_FISCAL_CODE unreachable in the default image

[GiorgioDotcom]

Two related behaviors observed on presidio-analyzer 2.2.364 (ghcr.io/data-privacy-stack/presidio-analyzer:2.2.364, default configuration, language en).

1. Unsupported entity silently ignored

   POST /analyze
   {"text":"codice fiscale RSSMRA85M01H501Q, IBAN IT60X0542811101000000123456","language":"en","entities":["IT_FISCAL_CODE","IBAN_CODE"]}

   -> 200, [{"entity_type":"IBAN_CODE", ...}]

IT_FISCAL_CODE is not in the response and nothing signals that it could not be served. Compare:

{"text":"...","language":"en","entities":["IT_FISCAL_CODE"]}

-> 500, {"error":"No matching recognizers were found to serve the request."}

So the caller gets an error only when the unsupported entity is the sole one requested. In a gateway or DLP integration this is a footgun: an operator adds IT_FISCAL_CODE to the block list, the service keeps answering 200, and fiscal codes flow through unscanned. I measured exactly this on two LLM gateways that embed Presidio (LiteLLM and GoModel): 11 out of 11 positions leaked the fiscal code while credit card, email and IBAN were blocked. Reproduction and raw results: https://github.com/GiorgioDotcom/llm-gateway-privacy-bench (risultati/gomodel-0.1.91-presidio-cf.json and risultati/litellm-1.98.0-presidio-cf.json; the one-line registry override is in presidio-cf/recognizers-cf.yaml, results with it in risultati/-riccf-.json).

Suggestion: return 400 when any requested entity has no recognizer for the requested language, or at least include the list of unserved entities in the response (or a warning header), so integrations can fail closed.

2. IT_FISCAL_CODE unreachable by default

ItFiscalCodeRecognizer exists and works well (it handles omocodia and validates the check character), but conf/default_recognizers.yaml registers it with supported_languages: [it], while the published image loads only the en NLP model and supported_languages: [en]. Result: with the image as shipped there is no way to detect Italian fiscal codes without overriding the registry.

The one-line override works (RECOGNIZER_REGISTRY_CONF_FILE pointing to a copy of default_recognizers.yaml where ItFiscalCodeRecognizer has supported_languages: [en]); after that GoModel with Presidio caught the fiscal code in every position, including omocodic codes.

Suggestion: either document this override next to the Italian recognizers, or register the pattern-based country recognizers (which do not depend on the NLP model) for en as well as their native language, as is already done for several other countries.

Happy to open a PR for the docs or the registry change if the maintainers prefer one of the two.

Related, same integration pattern: BerriAI/litellm#41265

[omri374]

Thanks! This is useful, and I agree that passing an unsupported entity should return an error. We would need to think about handling this in zero-shot recognizers like SLM/LLM as they don't necessarily specify the entity list a priori.

On registering pattern-based country recongizers, the intention is to avoid false positives from recognizers you never need. Presidio has lots of country-specific recognizers and if your the likelihood of having those in your data is zero then you'd prefer having it disabled, with an ability to enable using config.

[nikolas-sapa]

Opened #2259 for the first part: requested entities that no recognizer can serve now raise a ValueError naming them, and /analyze returns 400 instead of silently answering 200 with partial results.

Two notes on the design, in case you prefer a different direction:

• The check lives in RecognizerRegistry.get_recognizers() rather than in the REST layer, because integrations embed AnalyzerEngine directly (LiteLLM does, which is where the leak was measured), so a server-only check would leave those callers with the silent behavior.
• On your point about zero-shot recognizers: LMRecognizer requires supported_entities and filters requested entities against it, and generic consolidation adds GENERIC_PII_ENTITY to that list, so every entity a zero-shot recognizer can serve is already reported by get_supported_entities(). Ad-hoc recognizers passed in the request stay honored.

I kept the second part of the report (registering the pattern-based country recognizers for en vs. documenting the override) out of that PR to keep the change focused. Happy to follow up with a docs PR for RECOGNIZER_REGISTRY_CONF_FILE if documenting the override is the direction you prefer.

[omri374]

FYI, this is not backward compatible (might start throwing errors for existing implementations) so we might have to defer with it to the next major version. For now, maybe a warning would be better

[GiorgioDotcom]

Agreed on backward compatibility. A warning as the default, plus an opt-in strict flag on AnalyzerEngine (or an env var for the server) that turns it into an error, would already let DLP-style integrations fail closed without breaking anyone; the hard error can then become the default in the next major.

On the second point, documenting the RECOGNIZER_REGISTRY_CONF_FILE override next to the country recognizers is the right direction given the false-positive argument. I can open that docs PR.