Segment merge, durable serialization, and reusable execution infrastructure

[nnunley]

What

Eight commits, each one idea, resequenced so the branch can be reviewed a commit at a time.

Reading order

#	Commit	What to look for
1	test(leit_index): create test segment files with tempfile	Pre-existing cleanup, first because everything after it inherits the habit. The mmap tests composed temp paths from pid + clock + counter and deleted the file at the end of the test body; now TempDir gives uniqueness from an exclusive create by the OS and cleanup on drop, including on panic. Net −35 lines.
2	feat(leit_text): add analysis schema identity	The shared primitive. AnalysisSchemaId + schema-aware FieldAnalyzers accessors, factored out ahead of both consumers.
3	feat(leit_index): merge segments deterministically	Merge policy, input validation, deterministic doc-id remap planning, merge execution, and a scoring oracle proving a merged segment scores identically to its sources.
4	feat(leit_index): serialize merged segments durably	On-disk form for compressed segments, rebuilt block metadata, and postings-encoding validation on full open. Ends with a merge → serialize → reopen → rescore round trip.
5	docs: design reusable execution infrastructure	Design and staged plan, committed before the implementation so commits 6–7 read against a stated intent.
6	test(index): add reference oracle and allocation measurement	The apparatus, before any optimization: a naive reference execution index used as a correctness oracle (with a feature-boundary test that compiles a consumer crate to prove bench-internals does not leak), plus a scoped allocation counter so tests assert exact allocation budgets.
7	feat(index): reuse execution allocations across queries	The optimization: finish_into for top-k sinks, a reusable preplanned execution scratch on ExecutionWorkspace, and compressed decode scratch moved out of the cursor. Each step pinned by the counter from commit 6, results held constant by the oracle. Ends with recorded allocation baselines.
8	feat(query): typed query pipeline	plan_program / search_program.

On commit 7

Planner::plan_program lowers a UserQueryProgram directly to an ExecutionPlan; ExecutionWorkspace gains plan_program/search_program alongside the textual variants, and leit_index re-exports the builder types so consumers need no direct leit_query dependency.

Motivation: callers routing natural-language prose through the textual parser hit hard parse failures on operator-looking input — colons read as field qualifiers, stray parens. A typed program lets a front-end guarantee totality on arbitrary user input and treats the string syntax as one client among several. Downstream consumer: graphrag-rs builds UserQueryProgram from a total parser and calls search_program; that change rides separately in its own repo.

• The term arm of the textual lowering is extracted into a shared helper, so field resolution, default-field expansion, and boost * default_boost composition are literally the same code on both paths. Parity is pinned by plan-equality and end-to-end hit-equality tests.
• Same max_depth/max_nodes guards as the textual path. Depth uses an iterative tri-color DFS with memoized depths: linear on shared-child DAGs, rejects over-deep chains mid-traversal instead of overflowing, and rejects cyclic hand-built arenas.
• Boost factors are validated at the typed boundary — NaN, infinite, and negative factors and composed-product overflow reject with QueryError::InvalidBoost. The textual path cannot produce these inputs, so this surface is typed-path-only.
• Phrase nodes lower to AND of their terms; Phase 1 has no positional data, so slop is carried but not enforced. The doc comment records the cross-field caveat explicitly: when positions land, lowering must OR per-field phrase nodes rather than just swap the AND node.

The textual parser is untouched.

Verification

• Every gate at every one of the eight commits, 8/8 clean: cargo fmt --all --check, taplo fmt --check, cargo clippy --workspace --all-targets --all-features --locked, and cargo test --workspace --locked --all-features. The branch is bisectable and each commit satisfies the repo's formatting and lint rules on its own, not just at the tip.
• Tip: 524 tests pass. prek run --all-files --hook-stage pre-push: 21/21 hooks pass, including cargo fmt, taplo fmt, all four clippy variants, --doc, and cargo doc.

Changes since the previous push

Relative to the previous push of this branch (03083893), three fixes, each folded into the commit that introduced the affected lines rather than added on top:

1. cargo fmt --check failed on the previous head — pub use ordering in leit_index/src/lib.rs plus call formatting in planner.rs, program_search.rs, plan_program.rs.

2. taplo fmt --check failed — three feature arrays needed sorting in leit_wind_tunnel and leit_wind_tunnel_query.

3. A real test bug in reference_feature_boundary: the temp directory was built by hand from pid + as_nanos(), but both tests in that file run in parallel in the same process, so when they started inside one clock tick they got the identical path — one test then read the other's main.rs while the other's Drop deleted the tree mid-build. Now uses tempfile::TempDir, so uniqueness comes from an exclusive create by the OS rather than from a name predicted out of the process id and the clock, and cleanup survives a panicking test. tempfile was already in Cargo.lock transitively via proptest, so this adds no new crate to the build graph. Separately, the nested generate-lockfile call now sets its own CARGO_TARGET_DIR instead of inheriting an ambient one.

   The same hand-rolled helper existed in leit_index/src/segment_format/mmap.rs on main. That is fixed here too, in commit 1.

Provenance

Implemented and reviewed with LLM assistance under red-first TDD; an independent cross-model review produced three robustness findings in the typed query work — parser recursion bound, DAG traversal memoization, boost validation — all fixed and covered by tests here.