fix(knowledge): bound distinct FTS terms per request, not per candidate fetch - #2382
fix(knowledge): bound distinct FTS terms per request, not per candidate fetch#2382ohdearquant wants to merge 1 commit into
Conversation
…te fetch A knowledge.search query with many distinct terms turned one request into one FTS MATCH statement per term, bounded only by the request read deadline. Cap the number of distinct scoreable terms one request can spend on full-text lookups, and share that budget across every lexical fetch a single request makes: query decomposition can call the fetch up to three times (the full query plus two sub-queries), and without sharing, each of those would get its own allowance and triple the effective cap. candidate_provenance.terms_truncated reports when a query exceeded the budget. The raw existence probe that distinguishes a true miss from a match removed by status/kind filtering now scopes itself to the same (possibly truncated) term set the candidate fetch searched, so a truncation-caused miss is never mislabeled as filtered.
|
Measured against the current served build, before this lands as the fix for the reported Term count does not drive the flag on the search path. A nested-prefix ladder, holding the The cap would not bind the reported workload. The queries in the incident expand to about 20 What does vary is how much the stage reads. A single search returning three results was None of this argues against the change on its own terms: bounding long-query fan-out is a real |
|
Correction to my previous comment, on one sentence in it. I wrote that a single search returning three results was measured making the serving process pull The rest of that comment stands. The term ladder, the exclusion of carrier and result limit, and The narrow conclusion is unchanged and does not depend on the retracted sentence: this change |
Bounds distinct FTS terms per request rather than per candidate fetch.
FTS_TERM_COUNT_LIMIT(32) is a request-wide budget carried on the search context, shared by the full query and both decomposed sub-queries, so a decomposed search never fans out to three times the limit.no_matchwithterms_truncated: true, never asfiltered.Stacked on the score-provenance change; targets that branch until it merges.