Warn about unsupported requested entities while preserving partial results - #2259
Open
nikolas-sapa wants to merge 2 commits into
Open
nikolas-sapa wants to merge 2 commits into
nikolas-sapa wants to merge 2 commits into
Conversation
The analyzer silently ignored requested entities that no recognizer could serve in the requested language, returning 200 with partial results. In gateway/DLP setups this fails open: an operator adds an entity to the block list, the service keeps answering successfully, and that entity is never scanned. The behavior was inconsistent too, since requesting only unsupported entities already raised an error. RecognizerRegistry.get_recognizers now collects the requested entities that have no matching recognizer for the language and raises a ValueError listing them, so the failure is explicit and actionable. The REST API maps the ValueError to a 400 response instead of a 500. Zero-shot recognizers are unaffected: LMRecognizer requires supported_entities and filters requests against them, and generic consolidation adds GENERIC_PII_ENTITY to that list, so every entity a zero-shot recognizer can serve is reported by get_supported_entities. Note for the reviewer: the behavior change also affects two existing engine tests which used a not-yet-registered entity; they now assert the raised error, which makes the documented contract explicit.
nikolas-sapa
force-pushed
the
fix/error-on-unsupported-entities
branch
from
September 16, 2026 12:32
890afe5 to
31c887a
Compare
Collaborator
|
As disccused on #2256, this is not backward compatible. It's best if this is done in stages, where the first stage is a warning + a notice that this would raise a warning in future versions. wdyt? |
nikolas-sapa
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refs #2256. Following the maintainer request for a staged change, preserve the existing behavior for requests containing both supported and unsupported entities.
The existing registry warning now explains that ignoring unsupported entities is deprecated and will raise an error in a future version. It identifies the entity and language, and directs callers to AnalyzerEngine.get_supported_entities(language) or a matching recognizer. No release date or configuration option is introduced.
Compatibility
Verification
The full analyzer suite, model-dependent tests, and live-server E2E suite were not run. E2E expectations were aligned with the preserved API behavior; those changes are not claimed as live-server verified.
Scope
No country-recognizer language configuration changes, new strictness option, or changelog edits. This supersedes the initial hard-error implementation.