Fix broken cursor pagination for word_similarity relevance search - #201
Merged
BarryArinze merged 1 commit intoAug 21, 2026
Merged
Conversation
Cursor pagination for searchCampaignsByRelevance/searchDonationsByRelevance had two real bugs, not just the float precision issue: the WHERE clause referenced the SELECT-list alias "score" in `(score, id) < (...)`, which Postgres rejects outright (verified against a live Postgres — any second page request threw "column \"score\" does not exist"); and even once fixed to reference the expression directly, comparing the float4 word_similarity output against a JS number bound as `numeric` silently dropped or duplicated rows, including exact-tie rows that share a score (confirmed: an equality check between a real column and its own value round-tripped through Prisma returned zero matches). Fix: cast the score expression to double precision everywhere it's used (SELECT, threshold filter, cursor comparison) so every comparison happens in one consistent Postgres type instead of relying on implicit float4/ numeric promotion, and decompose the cursor condition into an explicit `score < x OR (score = x AND id < y)` instead of a tuple comparison, so it never depends on the "score" alias being visible in WHERE. Also add COALESCE around the description word_similarity call for campaigns, matching the existing donation search handling of NULL columns. Cursors are now versioned (`v: 2`) so a pre-fix cursor is detected and treated as absent (falls back to page one) instead of being silently misinterpreted with the old, broken comparison semantics. Validated the SQL-level fix against a real Postgres 15 + pg_trgm instance: 20 identically-scored rows paginate across two pages with zero duplicates and zero gaps, and a 50-row match set walks cleanly across all pages.
Petah1
force-pushed
the
fix/194-cursor-pagination-word-similarity
branch
from
August 20, 2026 13:14
bf467ec to
abef4f7
Compare
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
closes #194. Cursor pagination for
searchCampaignsByRelevance()/searchDonationsByRelevance()insrc/services/search.service.tshad two real bugs, not just the float-precision issue described in the issue:scoredirectly (WHERE (score, id) < (...)). Postgres does not allow referencing an output alias from the same query's WHERE clause — verified against a live Postgres 15 instance that any second-page request threwcolumn "score" does not exist.word_similarity()returnsfloat4. The cursor's score, once round-tripped through Prisma into a JS number and sent back as a query parameter, gets bound asnumeric. Comparing afloat4column against anumericparameter silently produces wrong results — confirmed directly against Postgres that an equality check between arealcolumn and its own round-tripped value returned zero matching rows.Fix
double precisioneverywhere it's used (SELECT,> 0.2threshold filter, cursor comparison), so every comparison happens in one consistent Postgres type instead of relying on implicit float4/numeric promotion.(score, id) < (x, y)with an explicitscore < x OR (score = x AND id < y), which also removes the alias-in-WHERE dependency causing the crash.COALESCE(word_similarity(query, description), 0)for campaigns, matching the existing NULL-handling already used in donation search.v: 2). A pre-fix (unversioned) cursor is now detected and treated as absent — falls back to page one — instead of being silently misinterpreted with the old, broken comparison semantics.Applied identically to both
searchCampaignsByRelevance()andsearchDonationsByRelevance(). Beneficiary search and the non-relevance sort paths are untouched (out of scope per the issue).Test plan
src/services/search.service.test.ts: cursor encode/decode roundtrip for a float4-range value, legacy/unversioned cursor is dropped gracefully, adversarial manipulated cursor (score: -1) doesn't crash or leak rows, generated SQL usesCAST(... AS DOUBLE PRECISION)andCOALESCEfor description.tests/performance/search.performance.test.tsagainst a real Postgres 15 +pg_trgminstance (validated locally with docker):word_similarityscore: partition cleanly across two pages byid DESC, zero overlap.npx tsc --noEmitclean for the changed file.searchBeneficiaries/globalSearchtests) predate this change and are unaffected.