Skip building indexes for probers that will only be probed once

[yihozhang]

Profiling shows >50% of the indices are only built and then probed only once, so it is wasteful. This PR avoids building the index when this happens, and instead just scans the subset.

It also adds an index_stats module and a compile flag that, when enabled, shows the stats of indices built.

Also updated bench.py to interleave the execution when benchmarking.

Results:

# Benchmark diff

  Generated  : 2026-07-18T15:42:14
  Baseline   : 56b84600 (56b84600db70)  —  Merge pull request #948 from egraphs-good/perf-sorted-column-index
  Comparison : 64e8fe39 (64e8fe3953d0)  —  interleave run

  Benchmark                                  Before (s)   After (s)     Δ (s)      Δ %
  ────────────────────────────────────────────────────────────────────────────────────────
  hardboiled_conv1d_32.egg                        0.134       0.139  +  0.005  +   3.4%  ▲ slower
  hardboiled_conv1d_128.egg                       0.340       0.355  +  0.015  +   4.5%  ▲ slower
  luminal-llama.egg                               0.083       0.081   -0.002    -1.9%  ▼ faster
  python_array_optimize.egg                       0.266       0.230   -0.035   -13.3%  ▼ faster
  cykjson.egg                                     0.033       0.033  +  0.000  +   0.1%  ·
  eggcc-extraction.egg                            0.213       0.215  +  0.002  +   0.9%  ▲ slower
  llama.egg                                       0.435       0.415   -0.021    -4.7%  ▼ faster
  paged_llama.egg                                 1.679       1.686  +  0.007  +   0.4%  ·
  qwen.egg                                        0.485       0.455   -0.030    -6.1%  ▼ faster
  qwen3_moe.egg                                   0.678       0.642   -0.035    -5.2%  ▼ faster
  whisper.egg                                     1.513       1.374   -0.139    -9.2%  ▼ faster

  Summary: 6 faster  ·  3 slower  ·  2 unchanged  ·  0 missing
  Overall average Δ: -2.84%

  VERDICT: IMPROVEMENT

I am not too sure how to eliminate the regression.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 96.15385% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.31%. Comparing base (56b8460) to head (64e8fe3).

Files with missing lines	Patch %	Lines
core-relations/src/free_join/execute.rs	96.04%	7 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #953      +/-   ##
==========================================
+ Coverage   86.25%   86.31%   +0.06%     
==========================================
  Files          95       96       +1     
  Lines       29269    29410     +141     
==========================================
+ Hits        25245    25385     +140     
- Misses       4024     4025       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[codspeed-hq]

Merging this PR will improve performance by 24.97%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 36 untouched benchmarks
⏩ 227 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	tests[python_array_optimize]	1.4 s	1.1 s	+24.97%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing skip-1-prober (64e8fe3) with main (56b8460)}

Footnotes

1. 227 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[ezrosent]

What about the regressions for hardboiled?

[yihozhang]

@ezrosent I'm not exactly sure about what leads to the regression (the regression seems to be consistent for this benchmark). My suspicion is that this is due to the atomic counter and the additional checking, and maybe the more bloated run_plan. I can look deeper if you think this can be further improved!