feat(mem_wal): serve single-column and nested multi-match on the fresh tier

[hamersaw]

Problem

The fresh tier served exactly one multi-match shape: a top-level query naming two or more distinct columns, which fts_plan_shape decomposes into one arm per leaf before any validation runs. Everything else was refused:

• A multi-match naming a single distinct column — MultiMatchQuery("alpha", ["body"]), or ["body", "body"] with two boosts — took the columns.len() <= 1 early return into the Bound path and was rejected whole by validate_lsm_fts_query (mem_wal/scanner/fts_search.rs:243).
• A multi-match nested in a boolean or boost was rejected by the memtable's query mapping (memtable/scanner/builder.rs:350, :431) at any depth.

The base table scores all of these as DisjunctionScorer(children, DisjunctionScore::Max) (lance-index/src/scalar/inverted/compound.rs:714), so a WAL-backed table answered with a hard error where its base-only twin answered correctly. On the LanceDB side that surfaced as a retryable 503 for ordinary client calls such as MultiMatchQuery("alpha", ["body"]) and MultiMatchQuery(...) & MatchQuery(...) (lancedb/sophon#7899).

Changes

fts_plan_shape decomposes every top-level multi-match per leaf. The MultiMatch check moves ahead of the column count, so one leaf per column (cross-column), several leaves on one column, and any mix all plan as independent arms unioned and collapsed to the best hit per row (FirstByPkExec). The single-arm case already short-circuits the union. The row-documents-only rule moves with it: collapsing per primary key drops list elements however many columns the leaves name.

FtsQueryExpr::MultiMatch — a best-child node evaluated by combine_multi_match, alongside combine_boolean / combine_boost in the same clause algebra, so it evaluates identically whether its leaves all come from one index or route to one index per column via search_cross_column. to_local_expr and the granularity visitor map an index-level multi-match onto it; validate_lsm_fts_query applies its per-leaf fuzzy/AND rule to the leaves instead of refusing the shape. The node participates in columns(), has_unbound_leaf(), bind_unbound_leaves(), and the boost/column pass-throughs like the other compound nodes.

Semantics

Best child per document, matching the compound scorer's DisjunctionScore::Max for a multi-match — not the sum a SHOULD would give. A leaf repeated with a higher boost dominates; a row matching several leaves comes back once at its highest score. Ranking parity between the fresh tier and committed data for every multi-match shape.

Tests

• test_multi_match_scores_best_child — evaluator keeps the best child, not the sum.
• multi_match_plans_one_arm_per_leaf_even_on_one_column — fts_plan_shape on ["text", "text"].
• single_column_multi_match_scores_each_row_by_its_best_leaf — planner: two arms on one column, each row scored exactly as the dominating leaf alone.
• nested_multi_match_reaches_the_active_memtable — MUST [multi_match(title, body)] MUST_NOT [title:spam] returns rows matching in either column, once, minus the excluded one.
• The memtable mapping test that asserted rejection now asserts the mapped tree.

cargo test -p lance --lib mem_wal: 727 passed. Clippy clean on the touched files.

🤖 Generated with Claude Code

[lance-gatekeeper]

❌ Gate recommendation: request changes.

The best-child evaluator matches the committed scorer, but top-level single-column multi-match queries should stay on the bound path. Reserve per-leaf union/collapse for queries that actually span columns so the new support also works on MemWAL tables without a primary key and preserves single-index document semantics.

[lance-gatekeeper]

Routing every top-level MultiMatch here to PerColumn makes even the one-arm MultiMatchQuery("lance", ["text"]) hit the cross-column primary-key guard in plan_search. MemWAL explicitly supports FTS without a primary key, and this query has nothing to collapse, so the advertised single-column shape still returns NotSupported. Determine distinct columns before this branch and keep single-column multi-match on Bound (where the new FtsQueryExpr::MultiMatch can evaluate it); use per-leaf plans only when the query truly spans columns.

Reproducer

In active_filtered_search_without_pk_applies_small_limit_after_filter, replace the existing Match query with:

FullTextSearchQuery::new_query(IndexFtsQuery::MultiMatch(
    MultiMatchQuery::try_new(
        "lance".to_string(),
        vec!["text".to_string()],
    )
    .unwrap(),
))

Then run cargo test -p lance --lib active_filtered_search_without_pk_applies_small_limit_after_filter -- --nocapture. The existing test should still return IDs [1, 2]; instead planning fails at fts_search.rs:620 with cross-column full-text search requires a primary key.