fix(mem_wal): build the transient FTS index with positions

[hamersaw]

Problem

A phrase query over a column with no FTS index silently returns nothing from the fresh tier while a plain match on the same rows succeeds.

The base answers that query: plan_phrase_query's no-index arm plans a flat scan, where token positions are implicit. The fresh tier serves an unindexed column through a transient index built over the visible prefix (transient_fts_index_store), and with no persisted index to inherit from it takes InvertedIndexParams::default() — whose with_position is false. search_phrase_tokens then returns Vec::new() for any multi-token phrase (index/fts.rs, "without them phrase search is unsupported, as on disk"). So every phrase hit in un-compacted rows is dropped, silently, on exactly the tables where the fresh tier exists to make those rows visible.

Verified against a live WAL pod (lancedb/sophon): unindexed column, match body:alpha → [1, 2]; phrase body:'alpha prose' → [].

Change

When no persisted index covers the column, the transient index is built with positions. The persisted-index case is untouched — it still inherits the index's analyzer and positional settings, which is the contract that keeps active rows agreeing with base and SSTable rows. The transient index lives for one query over the visible prefix, so the extra position storage is bounded by that prefix.

Test

phrase_over_an_unindexed_column_reaches_the_active_memtable: an active memtable with a PK index and no FTS index; a match over alpha returns rows 1 and 2 (precondition: the transient path serves the column), and the phrase alpha prose returns row 1 only — row 2 carries the terms reversed.

cargo test -p lance --lib mem_wal::scanner::fts_search: 41 passed.

Sibling of #9363 (multi-match shapes on the fresh tier); independent of it.

🤖 Generated with Claude Code

[lance-gatekeeper]

✅ Gate recommendation: approve.

The fix aligns the transient fresh-tier index with the base table’s unindexed phrase-query semantics while leaving persisted-index analyzer and positional settings unchanged. The added regression covers the no-index path and distinguishes ordered phrases from reversed terms. Reusing the transient index also keeps phrase and compound queries on the existing evaluator instead of adding a separate memtable flat-scan path.