fix(recommendations): cap hnsw.ef_search at pgvector's maximum

[blurbery]

Problem

Related issue: N/A
Validation tasks: changes #1145 C1, which generates recommendations and hasn't been run yet.

Recommendation candidate scans fail outright when they ask pgvector for more than 1,000 candidates. hnswEfSearch raises small scans to an hnsw.ef_search of 200 but never caps large ones, and pgvector rejects anything above 1,000:

ERROR: 1200 is outside the valid range for parameter "hnsw.ef_search" (1 .. 1000) (SQLSTATE 22023)

set_config fails, so the whole candidate query never runs. Watch Tonight's discover with a genre filter asks for five times its pool, which is 240 × 5 = 1,200 at the default limit (watch_tonight_cards.go). The handler logs a warning and drops its taste-profile candidates, so the cards fall back to the other sources. For You cluster rows hit the same error once their limit passes 1,000.

This clamps hnsw.ef_search to 1,000.

#1966 (draft) contains the same clamp, as part of a much larger change. This is the one-line fix with a regression test, so main stops failing now. If this merges first, #1966 can take main's version on rebase.

Approach

• hnswEfSearch returns min(max(limit, 200), 1000). With hnsw.iterative_scan = relaxed_order, which every caller sets, the index keeps returning rows past ef_search. I checked that locally: with ef_search 1000 and LIMIT 1200, the HNSW index scan read 2,105 index entries rather than stopping at 1,000.
• Nothing else changes. The ANN limits stay as they are (up to 2,000).

Validation

• New DB contract TestFindTasteProfileCandidatesWithGenresAboveEfSearchMaximumDB calls FindTasteProfileCandidates with a genre and a pool of 240, on connections that have already loaded pgvector, as a reused pooled connection has. On a connection that hasn't, PostgreSQL accepts the value, then only warns when pgvector loads and falls back to the default ef_search of 40, which the clamp also avoids. On main the test fails with the SQLSTATE 22023 error above. On this branch it passes and returns the seeded candidates. Added to scripts/ci/db-contracts.txt.
• TestHNSWEfSearchStaysWithinPgvectorRange (renamed from TestHNSWEfSearchUsesCandidateLimitFloor) adds 1000, 1001, 1200 and 2000.
• go test ./internal/recommendations/ against a migrated PostgreSQL 18 database with pgvector 0.8.7 passes. make lint-changed reports 0 issues.

Benchmarks

Not applicable: this turns an error into a working query and changes no query plan.

Evidence

The same set_config call against my server's database (read-only transaction, pgvector 0.8.2, after loading the extension in the session):

SELECT set_config('hnsw.ef_search','1000',true);  -- 1000
SELECT set_config('hnsw.ef_search','1200',true);  -- ERROR:  1200 is outside the valid range for parameter "hnsw.ef_search" (1 .. 1000)

My server's retained logs (since its last restart on 7 October) don't contain this error. So I can't show a before/after of a real Watch Tonight response. What users see when it does happen is a genre-filtered Watch Tonight without its taste-profile picks. The after evidence is the regression test above. Its raw output:

Tests on main and on this branch

== main's repo.go and accuracy_test.go
--- FAIL: TestFindTasteProfileCandidatesWithGenresAboveEfSearchMaximumDB (0.03s)
    hnsw_ef_search_db_test.go:65: FindTasteProfileCandidates with genres and limit 240: find taste profile candidates: configure hnsw candidate scan: ERROR: 1200 is outside the valid range for parameter "hnsw.ef_search" (1 .. 1000) (SQLSTATE 22023)
FAIL
== this branch
ok  	github.com/Silo-Server/silo-server/internal/recommendations	0.301s
== go test ./internal/recommendations/ against the database
ok  	github.com/Silo-Server/silo-server/internal/recommendations	0.604s

== a connection that has not loaded pgvector yet (main's behaviour)
SELECT set_config('hnsw.ef_search', '1200', true);   -> 1200
SELECT '[1]'::vector;
WARNING:  1200 is outside the valid range for parameter "hnsw.ef_search" (1 .. 1000)
SHOW hnsw.ef_search;                                  -> 40

== iterative scan with ef_search 1000 and LIMIT 1200 (3,000 random embeddings, HNSW index forced)
 Limit (actual rows=1034.00 loops=1)
   ->  Nested Loop (actual rows=1034.00 loops=1)
         ->  Index Scan using idx_media_item_embeddings_hnsw on media_item_embeddings e (actual rows=2105.00 loops=1)
               Order By: ((embedding)::halfvec(3072) <=> ($1)::halfvec(3072))

Risks

• Scans that asked for more than 1,000 candidates now start with an ef_search of 1,000 and rely on the iterative scan for the rest. They previously failed.
• fix(recommendations): make the recommendation pipeline correct, isolated and observable #1966 changes the same lines. Whichever merges second has a trivial conflict.

Checklist

• I read and can explain the complete diff.
• This pull request addresses one concern.
• The Evidence section shows every change a user can see, or says there is none.

AI Disclosure

• Harness: Claude Code (desktop app)
• Tool(s): Claude Code
• Model(s): claude-opus-5-5
• Involvement: AI-assisted
• Adversarial review: n/a

AI-assisted with Claude Opus. I directed the task and designed the work.

[silo-kody]

Silo Kody — review complete

Review finished. Check the inline comments for findings and verify each suggestion against the code and tests.

Reviewing changes in Silo

• Include the related issue, expected behavior, and validation steps in the PR description.
• For API changes, describe the effect on Apple and Android clients and Jellyfin compatibility.
• For plugin changes, identify the affected SDK contract, plugin, and catalog entry.
• Follow this repository's AGENTS.md and CONTRIBUTING.md.
• Request another review with @kody start-review in a PR comment.
• React with 👍 or 👎 to give feedback on individual suggestions.

Review settings

Review Options

The following review options are enabled or disabled:

Options	Enabled
Bug	✅
Performance	✅
Security	✅
Business Logic	❌

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 18219819-bfc5-4806-8a19-7af4b4ca7c4c

📥 Commits

Reviewing files that changed from the base of the PR and between ca186fe and a489015.

📒 Files selected for processing (4)

• internal/recommendations/accuracy_test.go
• internal/recommendations/hnsw_ef_search_db_test.go
• internal/recommendations/repo.go
• scripts/ci/db-contracts.txt

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

Recommendation candidate scans now cap hnsw.ef_search at 1000 while retaining a minimum of 200. Unit and database tests cover values at and above the maximum.

Changes

Recommendation candidate search

Layer / File(s)	Summary
Bound the ef_search setting internal/recommendations/repo.go, internal/recommendations/accuracy_test.go	hnswEfSearch now clamps values to 200–1000. Unit tests check that the maximum is retained and larger values are capped.
Validate candidate scans in the database internal/recommendations/hnsw_ef_search_db_test.go, scripts/ci/db-contracts.txt	A database test checks that a candidate scan with a pool size of 240 returns three matching Drama items. The test is added to the database contract list.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: quick104

Merge Risk: ⚪ Minimal · up to a4890

The ef_search cap and its boundary tests present no identified merge-blocking issue; merge after normal checks.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 3 files. (1 skipped: 1…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely describes the main change: capping hnsw.ef_search at pgvector's maximum for recommendations.
Description check	✅ Passed	The description directly explains the pgvector error, the clamp implementation, affected recommendation scans, regression tests, validation, and known validation limits.

Full details: Docstring Coverage

Explanation

Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 3 files. (1 skipped: 1 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}