Problem
Prose search currently sanitizes each whitespace-delimited token and joins the resulting FTS5 terms with spaces. SQLite FTS5 interprets that form as an implicit AND.
This is precise for short literal queries, but brittle for conversational recall. If a user includes several plausible synonyms—such as “adaptation”, “implementation”, “dialect”, and “variant”—the historical message must contain every surviving term. The correct record can disappear even when it contains the core entities and requested concept.
Users and agents currently recover by manually shortening and retrying the query. That adds latency, and each retry can introduce additional query-echo rows into the active session.
Proposed Solution
Add an explicit, measurable query-relaxation mode and consider a conservative progressive fallback for zero-result searches.
Recommended sequence:
- execute the current strict AND query;
- if it yields no usable results, remove low-information terms while preserving quoted phrases and identifiers;
- optionally execute an OR/minimum-match form with a documented threshold;
- rank strict matches above relaxed matches;
- report which relaxation stage produced each result.
Expose the behavior explicitly, for example through --match strict|relaxed or equivalent, before considering any default change. Quoted phrases, paths, error strings, flags, and --content-type tool searches should remain strict unless the caller opts in.
Acceptance should be driven by the conversational-recall cohort rather than anecdotal examples:
- relaxed recall@5 improves over the strict baseline;
- exact/error/tool-query precision does not regress;
- p95 latency remains interactive;
- output identifies relaxed results so callers do not mistake them for exact matches.
Alternatives Considered
- Use OR for every query. Rejected because it would increase noise substantially and weaken exact command/error retrieval.
- Tell callers to shorten queries manually. This is the current workaround and requires multiple round trips.
- Enable embeddings without a benchmark. Semantic search may help, but it adds operational cost and should be justified against the same eval cohort.
- Silently discard arbitrary terms. Rejected because the behavior would be difficult to reason about and reproduce.
Additional Context
internal/storage/search.go:sanitizeFTS5Query currently constructs the implicit-AND expression. The repository's existing eval-set should be extended with conversational paraphrase and query-echo cases before this behavior changes.
This issue depends conceptually on the recall evaluation enhancement and should preserve strict search as an explicit, stable mode.
Problem
Prose search currently sanitizes each whitespace-delimited token and joins the resulting FTS5 terms with spaces. SQLite FTS5 interprets that form as an implicit AND.
This is precise for short literal queries, but brittle for conversational recall. If a user includes several plausible synonyms—such as “adaptation”, “implementation”, “dialect”, and “variant”—the historical message must contain every surviving term. The correct record can disappear even when it contains the core entities and requested concept.
Users and agents currently recover by manually shortening and retrying the query. That adds latency, and each retry can introduce additional query-echo rows into the active session.
Proposed Solution
Add an explicit, measurable query-relaxation mode and consider a conservative progressive fallback for zero-result searches.
Recommended sequence:
Expose the behavior explicitly, for example through
--match strict|relaxedor equivalent, before considering any default change. Quoted phrases, paths, error strings, flags, and--content-type toolsearches should remain strict unless the caller opts in.Acceptance should be driven by the conversational-recall cohort rather than anecdotal examples:
Alternatives Considered
Additional Context
internal/storage/search.go:sanitizeFTS5Querycurrently constructs the implicit-AND expression. The repository's existing eval-set should be extended with conversational paraphrase and query-echo cases before this behavior changes.This issue depends conceptually on the recall evaluation enhancement and should preserve strict search as an explicit, stable mode.