Skip to content

Revert draft BM25 ranking changes from PR #54 #56

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
TeoSlayer wants to merge 1 commit into from
Closed

Revert draft BM25 ranking changes from PR #54 #56

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/ENV.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,13 +153,13 @@ not re-enable them without the confidentiality decision in that section.
| `COSIFT_BM25_K1` | float | `1.2` (PebbleBM25 default) | BM25 term-frequency saturation `k1`; applied only if parseable and `> 0`. | `serve_search.go:1350` |
| `COSIFT_BM25_B` | float | `0.75` (PebbleBM25 default) | BM25 length-normalization `b`; applied only if parseable and `> 0`. | `serve_search.go:1358` |
| `COSIFT_BM25_MIN_IDF` | float | `0.5` | IDF floor below which a query term is dropped as a stopword. `0` disables pruning. Must be `>= 0`. | `internal/index/pebble_bm25.go:38` |
| `COSIFT_BM25_DISABLE_MAXSCORE` | bool (non-empty disables) | unset → MaxScore optimization **enabled** | Any non-empty value disables the WAND/MaxScore early-termination optimization for benchmark-grade lossless ranking. Phrase queries skip the optimization unconditionally. | `internal/index/pebble_bm25.go:230` |
| `COSIFT_BM25_DISABLE_MAXSCORE` | bool (non-empty disables) | unset → MaxScore optimization **enabled** | Any non-empty value disables the WAND/MaxScore early-termination optimization for benchmark-grade lossless ranking. | `internal/index/pebble_bm25.go:223` |
| `COSIFT_BM25_TOPK_POOL_FACTOR` | int | `50` | Sizes the metadata-resolution pool at `factor*k` candidates (PebbleBM25 top-k pool). Must be `>= 1`. Raise if the pool-cap log line fires on ranking-sensitive traffic. | `internal/index/pebble_bm25.go:50` |
| `COSIFT_BM25_DISABLE_TOPK_POOL` | bool (non-empty disables) | unset → top-k pool **enabled** | Any non-empty value restores the resolve-all metadata path (every scored candidate gets a `GetDocMeta`) — the pre-pool behavior, for lossless A/B comparison. | `internal/index/pebble_bm25.go:294` |
| `COSIFT_BM25_DISABLE_TOPK_POOL` | bool (non-empty disables) | unset → top-k pool **enabled** | Any non-empty value restores the resolve-all metadata path (every scored candidate gets a `GetDocMeta`) — the pre-pool behavior, for lossless A/B comparison. | `internal/index/pebble_bm25.go:295` |
| `COSIFT_DEFAULT_DECAY_DAYS` | float | `180` (6-month half-life) | Default recency half-life (days) applied when a request has no explicit `?decay=`. `0` disables decay globally. Must be `>= 0`; explicit `?decay=N` still wins. | `serve_search.go:1592` |
| `COSIFT_AUTHORITY_ALPHA` | float | scorer's built-in default (`authority.New()`) | Authority-score blend weight; applied only if parseable and `>= 0`. **Also a BM25 scan-cost knob**: it sets the MaxScore pruning ceiling `maxMult = 1+alpha` (`internal/index/pebble_bm25.go:242`), so raising it prunes less (more posting scans, larger per-query score map) and lowering it prunes more. Lower alpha before reaching for `COSIFT_BM25_DISABLE_MAXSCORE`. | `serve_setup.go:135` |
| `COSIFT_TRANCO_CSV` | string (path) | unset → embedded whitelist + TLD heuristics only | Path to a Tranco rankings CSV to enrich authority scoring. | `serve_setup.go:140` |
| `COSIFT_MAJESTIC_CSV` | string (path) | unset → embedded whitelist + TLD heuristics only | Path to a Majestic Million CSV to enrich authority scoring. | `serve_setup.go:149` |
| `COSIFT_AUTHORITY_ALPHA` | float | scorer's built-in default (`authority.New()`) | Authority-score blend weight; applied only if parseable and `>= 0`. | `serve_setup.go:188` |
| `COSIFT_TRANCO_CSV` | string (path) | unset → embedded whitelist + TLD heuristics only | Path to a Tranco rankings CSV to enrich authority scoring. | `serve_setup.go:193` |
| `COSIFT_MAJESTIC_CSV` | string (path) | unset → embedded whitelist + TLD heuristics only | Path to a Majestic Million CSV to enrich authority scoring. | `serve_setup.go:202` |
| `COSIFT_DISABLE_ENTITY_EXPAND` | bool (non-empty disables) | unset → entity expansion **enabled** | When unset, query is expanded with canonical-attribute rewrites (`qexpand.RewriteEntity`). Any non-empty value disables it. | `serve_search.go:2064` |
| `COSIFT_HYDE_CACHE_SIZE` | int | `256` | Capacity of the HyDE hypothetical-document cache; must be `> 0` (warns on bad value). | `serve_setup.go:161` |
| `COSIFT_PARA_CACHE_SIZE` | int | `256` | Capacity of the paraphrase cache; must be `> 0` (warns on bad value). | `serve_setup.go:173` |
Expand Down
124 changes: 0 additions & 124 deletions docs/K-DEPENDENCE.md
View file
Open in desktop

This file was deleted.

20 changes: 0 additions & 20 deletions docs/TUNING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,26 +111,6 @@ set (same cost as before the pool, plus a small selection overhead).
`COSIFT_BM25_DISABLE_TOPK_POOL=1` restores the resolve-all path for lossless
A/B comparison; both vars are read per query, no restart needed.

### MaxScore pruning depth: `COSIFT_AUTHORITY_ALPHA`

The MaxScore early-break compares the k-th *raw* score against
`remainingMax * (1 + alpha)`, because a doc's final score is
`raw × authority multiplier` and that multiplier tops out at `1 + alpha`. So
alpha is a latency knob as well as a ranking knob: **higher alpha prunes less**
(queries whose `theta / remainingMax` lands in `[1, 1+alpha)` now scan the
lower-IDF term's full posting list and grow the per-query score map
accordingly); **lower alpha prunes more**.

If BM25 p50 regresses, lower alpha first — it narrows the non-pruning band
proportionally and keeps ranking sound. `COSIFT_BM25_DISABLE_MAXSCORE=1` is the
opposite of a mitigation: it removes pruning entirely and makes common-term
queries strictly slower. Alpha is read once at server start, so changing it
needs a restart (unlike the two pool vars above).

Phrase queries (`"like this"`) skip MaxScore unconditionally and pay the full
scan regardless of alpha: the threshold is a k-th over all scored docs, which
says nothing about the k-th of the phrase-filtered subset.

## Latency budget

### Don't enrich what you don't need: `?enrich=false`
Expand Down
109 changes: 31 additions & 78 deletions internal/index/pebble_bm25.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,46 +55,6 @@ func bm25TopKPoolFactor() int {
return 50
}

// bm25RankDepth floors the depth at which ranking decisions are made, so they
// stop depending on the caller's k.
//
// Two decisions are k-shaped. The MaxScore break compares against
// kthLargest(scores, k): a small k gives a higher theta, so the scan stops
// earlier and produces a different score map. The resolution pool is factor*k:
// a small k resolves fewer candidates. Measured on prod 2026-09-15 at factor
// 200, 51 of 60 goldens returned a different top-10 at k=10 than at k=50, and a
// median cap-bound query resolved 2,000 of ~68,500 in-band candidates.
//
// Running both at kEff = max(k, depth) and truncating afterwards makes the
// top-k of any k <= depth a prefix of the same ranking. The cost is that every
// query pays the depth-k price. 0 keeps the k-shaped behaviour.
func bm25RankDepth() int {
if v := os.Getenv("COSIFT_BM25_RANK_DEPTH"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n >= 0 {
return n
}
}
return 0
}

// bm25EffectiveK is the internal ranking depth for a caller asking for k.
func bm25EffectiveK(k int) int {
if k <= 0 {
return k
}
if d := bm25RankDepth(); d > k {
return d
}
return k
}

// bm25MaxScoreAuthorityBound reports whether the MaxScore early-termination
// bound accounts for the authority multiplier applied after the scan. 0 selects
// the pre-2026-09 behaviour, which prunes harder and is measurably less exact.
func bm25MaxScoreAuthorityBound() bool {
return os.Getenv("COSIFT_BM25_MAXSCORE_AUTHORITY") != "0"
}

// PebbleBM25 mirrors BM25 but reads from a PebbleStore.
type PebbleBM25 struct {
store *store.PebbleStore
Expand Down Expand Up @@ -134,14 +94,6 @@ func (b *PebbleBM25) WithAuthority(a *authority.Scorer) *PebbleBM25 {
return b
}

// maxAuthorityMult upper-bounds Scorer.Multiplier (= 1 + alpha*score, score in [0,1]).
func (b *PebbleBM25) maxAuthorityMult() float64 {
if b.authority == nil {
return 1.0
}
return 1.0 + b.authority.Alpha()
}

// WithBoost returns a shallow copy of b with the given docID→multiplier map
// applied post-scoring. Use this for site= queries: enumerate the site's
// docIDs, pass a 50× multiplier, and the site's docs will always appear in
Expand Down Expand Up @@ -262,28 +214,32 @@ func (b *PebbleBM25) Search(ctx context.Context, q string, k int) ([]Hit, error)
// The BM25 contribution of one term to any doc is bounded above by
// idf*(k1+1) — saturates as tf grows relative to docLen. Sum of these
// upper bounds across remaining terms = the maximum score a doc not
// yet seen can ever accumulate. Docs already in `scores` stop
// accumulating at the break too, so top-K membership is approximate,
// not lossless. COSIFT_BM25_DISABLE_MAXSCORE=1 disables the
// optimization for benchmark-grade lossless ranking. Phrase queries opt
// out: theta cannot threshold the phrase-filtered subset (empty results).
maxScoreEnabled := os.Getenv("COSIFT_BM25_DISABLE_MAXSCORE") == "" && len(phrases) == 0
kEff := bm25EffectiveK(k)
maxMult := 1.0
if bm25MaxScoreAuthorityBound() {
maxMult = b.maxAuthorityMult()
}
// yet seen can ever accumulate. If that sum drops below the current
// K-th best partial score, no future doc can enter top-K. Top-K
// membership stays lossless; in-top-K ranking can shift (the rerank
// pipeline re-orders anyway, and the score-decay/MMR stages tolerate
// approximate partial scores). COSIFT_BM25_DISABLE_MAXSCORE=1 disables
// the optimization for benchmark-grade lossless ranking.
maxScoreEnabled := os.Getenv("COSIFT_BM25_DISABLE_MAXSCORE") == ""
remainingMax := 0.0
if maxScoreEnabled {
for _, c := range active {
remainingMax += c.idf * (b.k1 + 1.0)
}
}
for i, c := range active {
// theta is raw; an unseen doc's final ceiling is remainingMax*maxMult.
if maxScoreEnabled && i > 0 && len(scores) >= kEff {
theta := kthLargest(scores, kEff)
if remainingMax*maxMult < theta {
// Pre-scan MaxScore check: decide whether term i (and everything
// after it, since active is sorted descending by IDF) can still push
// an unseen doc into top-K BEFORE paying to scan term i's postings.
// remainingMax here is the sum of max contributions of terms i..end
// (idf*(k1+1) each). A doc not yet in `scores` can gain at most
// remainingMax from the un-scanned terms; if that's below the current
// K-th best score, scanning i..end can only reorder within top-K —
// which the reranker fixes — so stop. This catches the common-term
// last-term full scan the old post-scan i==len-1 guard always paid.
if maxScoreEnabled && i > 0 && len(scores) >= k {
theta := kthLargest(scores, k)
if remainingMax < theta {
break
}
}
Expand Down Expand Up @@ -312,12 +268,13 @@ func (b *PebbleBM25) Search(ctx context.Context, q string, k int) ([]Hit, error)

// Apply per-doc boosts (e.g. site= queries) before sorting. Boosted docs
// already present in `scores` (i.e. they matched ≥1 query term) get the
// full multiplier on their (possibly MaxScore-truncated) partial. Boosted
// docs with zero term overlap are absent from `scores`; for small boost
// sets we seed them with a tiny base score (boostSeedBase) before
// multiplying so they still enter the candidate pool — landing at
// boostSeedBase*mult (e.g. 0.05), below any genuine single-term BM25
// match but enough for the reranker to judge them.
// full multiplier. Boosted docs with zero term overlap are absent from
// `scores`; for small boost sets we seed them with a tiny base score
// (boostSeedBase) before multiplying so they still enter the candidate
// pool — landing at boostSeedBase*mult (e.g. 0.05), below any genuine
// single-term BM25 match but enough for the reranker to judge them. This
// also closes the MaxScore early-termination gap: a boosted doc whose only
// matching posting list was skipped still gets seeded and surfaced.
//
// Seeding is gated by boostSeedMaxIDs: large boost sets (big sites) would
// inject a GetDocMeta per zero-overlap doc and flood the pool, so above
Expand All @@ -336,14 +293,7 @@ func (b *PebbleBM25) Search(ctx context.Context, q string, k int) ([]Hit, error)
// resolved; COSIFT_BM25_DISABLE_TOPK_POOL restores the resolve-all
// path (also taken for k<=0 = "return everything").
if k > 0 && os.Getenv("COSIFT_BM25_DISABLE_TOPK_POOL") == "" {
hits, err := b.resolveTopKPool(ctx, scores, phrases, kEff)
if err != nil {
return nil, err
}
if len(hits) > k {
hits = hits[:k]
}
return hits, nil
return b.resolveTopKPool(ctx, scores, phrases, k)
}

hits := make([]Hit, 0, len(scores))
Expand Down Expand Up @@ -392,7 +342,10 @@ const topKResolveSlack = 16
// the cap does bind (more than factor*k candidates inside that band) the
// truncation is logged — the operator signal to raise the factor.
func (b *PebbleBM25) resolveTopKPool(ctx context.Context, scores map[int64]float64, phrases []string, k int) ([]Hit, error) {
maxMult := b.maxAuthorityMult()
maxMult := 1.0
if b.authority != nil {
maxMult += b.authority.Alpha()
}

poolCap := bm25TopKPoolFactor() * k
pool := topCandidates(scores, poolCap)
Expand Down
Loading
Loading