search: recoverable payloads for Postgres-invalid regexes (#75) - #99
Merged
IceRhymers merged 1 commit intoJul 24, 2026
Conversation
A Postgres-invalid POSIX regex (e.g. /[/), negated or not, through search_code, symbol_search, or semantic_search's repo:/file: filters previously reached the database raw and escaped as an uncaught sqlalchemy.exc.DataError instead of a recoverable payload field. Adds a typed RegexInvalidError mapped by a single reraise_or_recoverable(error: DBAPIError) mapper shared across grep/symbols/ references/semantic, surfaced as a new additive `regex_invalid` field on the search_code and semantic_search envelopes, and mapped to a 400 with the Postgres message on the webui /api/search and /api/semantic routes. Refs #75
IceRhymers
merged commit Jul 24, 2026
bd51c16
into
integration/search-hardening-and-ci-integrity
4 checks passed
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refs #75
A Postgres-invalid POSIX regex (e.g.
/[/), negated or not, reachingsearch_code's grep/symbol legs,repo:/file:/sym:filter patterns, orsemantic_search'srepo:/file:filters previously escaped as an uncaughtsqlalchemy.exc.DataError(an unhandled 500-class fault at the MCP surface) instead of a recoverable payload field — violating this codebase's stated contract that "recoverable conditions are payload fields, never exceptions" (app/main.py's module docstring).app/search/errors.py: newRegexInvalidError;reraise_or_query_too_broad(OperationalError)replaced by a widerreraise_or_recoverable(error: DBAPIError) -> NoReturnmappingpsycopg.errors.QueryCanceled→QueryTooBroadError(unchanged) andpsycopg.errors.InvalidRegularExpression→RegexInvalidError(new); every otherDBAPIErrorre-raises unchanged (proven by identity assertions in tests).grep.py×2,symbols.py×2,references.py×3,semantic.py's RRF execute ×1 — newly wrapped, previously uncaught) switched toexcept DBAPIError+ the new mapper.app/service.py: new additive, permanentregex_invalid: str | Noneenvelope field.search_code_payloadgains a grep-leg trip arm (empty files,truncated=False— nothing was attempted-and-cut) and a symbol-leg-only trip arm (keeps grep's already-fetched files, mirroring the existingQueryTooBroadErrorpartial-result pattern).app/search/semantic.py: new_semantic_regex_invalid_payloadforrepo:/file:filter patterns Postgres rejects.webui/main.py:/api/searchand/api/semanticmapregex_invalidto a 400 with the actual Postgres message (previously/api/search500'd;/api/semanticnever had the field).tests/integration/test_grep.py'sxfail(raises=DataError, strict=False)pin (asserting the desired future behavior) is replaced with 3 gating tests.README.md,app/AGENTS.md,app/search/AGENTS.md,app/query/AGENTS.md,app/query/compiler.pyheader updated for the new field/contract.Out of scope (by design): issue #38 (Python-side regex CPU/ReDoS bounding, RE2, complexity gating) —
grep.py's "Uncapped Python CPU" caveat bullet is untouched verbatim.Test plan
make lint— ruff check + format + mypy: cleanmake test— 1076 passed (unit + observability)make test-integrationlocally against thecodesearch-pgPostgres container (tests/integration/test_grep.py,test_symbols_search.py,test_service.py): 80/80 passed, including newRegexInvalidErrorcoverage (bare/negated regex,repo:/sym:filter patterns, full-stack envelope). The repo's CI integration workflow (ci-lakebase.yml) is hard-gated onCI_LAKEBASE_ENABLED, unprovisioned repo-wide, so this local run is the only real integration evidence.test_migrations.py,test_reconcile.py,test_semantic_rrf.py,test_store_chunk_writer.py,test_webui_semantic.py,test_commit_search.py,test_mcp_server.py) are pre-existing environment gaps unrelated to this change (missinglakebase_tokenizer/lakebase_ann/lakebase_bm25extensions on vanilla local Postgres, and local env config) — confirmed identical on the unmodified base commit.