Use packed cached hashes for faster table mutations

[ezrosent]

Summary

This draft currently contains a 12-commit scalability stack. It improves action
execution, rebuilding, index construction, table mutation, union-find merging,
and parallel search; it also adds production-scheduler microbenchmarks and a
focused math scalability harness.

The final tree:

• stores predicted rows contiguously and allocates fresh IDs from
  execution-state-local reservations;
• hands rebuilding, index construction, union-find merging, and large join
  searches to workers at a coarser granularity;
• batches deletion work and avoids redundant key lookups when rebuild already
  knows the removed RowId;
• caches compact 32-bit probe hashes directly in non-partitioned mutation
  buffers; and
• does not retain the experimental PartitionedRowBuffer introduced
  midway through the stack.

The 12 commits

1. a89073c0 — Store predicted rows contiguously. Replaces the predicted
   value map's per-entry SmallVec/Vec ownership with a raw HashTable of
   hash/table/arity/backing-index metadata and one contiguous Vec<Value>.
   Collision checks borrow the actual key slice from that backing vector, which
   reduces allocation and indirection in lookup-or-insert actions.

2. c548da87 — Reserve fresh IDs per execution state. Replaces one shared
   atomic increment per fresh ID with disjoint ID ranges held by each
   ExecutionState. A global atomic reserves ranges, local allocation advances
   without synchronization, and unused suffixes are recycled through a queue;
   ordinary observable counters keep reservation size one.

3. 912ba559 — Explain counter reservation lifecycle. Documentation-only
   follow-up describing why reservations are disjoint, how unused tails are
   reused before extending the high-water mark, and how reservation size one
   preserves exact observable-counter semantics.

4. 149df0f1 — Build column indexes from parallel sorted runs. Replaces the
   serial scan plus many small locked handoffs used at high worker counts with
   coarse parallel scans. Producers create deduplicated (Value, RowId) runs
   sorted by destination shard; shard owners then bulk-build or merge those
   contiguous runs into dense or sparse buffered subsets.

5. 82e89543 — Use coarse partitions for table rebuilds. Coarsens both full
   and incremental rebuild work so a worker reuses its execution state,
   scratch space, and mutation buffers across a substantial partition. It also
   parallelizes the incremental dirty-ID scan, retains the prior 2K-row
   parallel loop at two to three workers, and uses serial handling where the
   input and worker-count thresholds favor it.

6. 4fcb49eb — Benchmark production table parallelism. Moves the table
   mutation microbenchmarks onto egglog's production work-stealing thread pool,
   adds the 12-P-core point, and adds a merge-only deletion case above the
   parallel threshold while remaining below the table-compaction threshold.

7. ab70a27e — Optimize parallel table deletion. Gives workers coarse
   contiguous shard ranges, separates random hash-table erasure from a batched
   stale-row write pass, and adds a known-RowId removal path so rebuilding
   does not reload and compare a key it has already resolved. These mechanisms
   remain in the final tree.

8. 35cbbfd7 — Benchmark large parallel table deletion. Adds a 4.2M-row,
   1.8M-removal merge-only case so each shard substantially exceeds L1 and
   deletion scaling can be measured without triggering compaction.

9. 6ebc2555 — Partition table mutations by cached hashes. Introduces the
   experimental PartitionedRowBuffer: producers cache full hashes and stably
   scatter rows into physical-shard/cache-window runs before consumers probe
   the table. It also adds bounded insert coalescing, direct merge/update,
   shard reservation, parallel unsorted compaction, and broader insertion
   benchmarks. The partitioned buffer and seal/scatter path are later removed
   by commit 12; the useful merge, compaction, and cached-probe building blocks
   remain.

10. b531eceb — Add math scalability benchmark harness. Adds
    scripts/math_scalability.py, which runs the production binary at
    1/2/4/8/12 threads, always passes --no-decomp, alternates measurement
    order, clears ambient egglog logging settings, and records wall time, CPU
    time, utilized cores, speedup, and efficiency as raw samples plus
    aggregates in JSON and aggregate rows in CSV. The selected egglog input
    controls the run count.

11. b818cd07 — Improve parallel union-find and join scalability. Adds a
    scoped concurrent union-find view using relaxed atomic loads/CAS, while
    preserving timestamp groups as semantic barriers and returning to ordinary
    serial access after workers join. It also extends coarse, work-stealable
    search partitioning to persistent indexes, scalar filtered indexes, and
    FusedIntersect, avoiding scan-and-project row-ID copies and secondary
    sorts.

12. 4a891241 — Use packed cached hashes for table mutations. Replaces the
    PRB experiment with producer-local, per-physical-shard HashedRowBuffers:
    one Vec<Value> stores rows in arrival order with a compact hash in the
    trailing lane. The 32-bit value preserves 25 low bucket-index bits and
    hashbrown's exact 7-bit H2 tag, while the full hash still chooses the
    physical shard. Table and known-removal entries pack hash plus RowId into
    eight bytes; equality checks still resolve compact-hash collisions.

Performance

All measurements used --no-decomp; math used (run 12).

Cumulative math uplift

The full 12-commit range was rerun locally as an exact, interleaved
c7fc1d77-versus-4a891241 A/B. Both freshly built binaries used the same
run-12 input. After one warmup per cell, a four-round Latin schedule repeated
twice balanced thread-count order, variant order, and pair position:

Threads	PR base mean	PR tip mean	Wall reduction	Paired speedup [95% bootstrap CI]
1	7.106s	6.164s	13.3%	1.153x [1.144x, 1.161x]
12	2.200s	0.978s	55.6%	2.250x [2.214x, 2.285x]

At 12 threads, total CPU falls from 14.80 to 7.25 CPU-seconds (51.0% less)
while effective utilization rises from 6.73 to 7.42 cores. One-to-12-thread
scaling improves from 3.23x to 6.30x, or from 26.9% to 52.5% parallel
efficiency. The improvement therefore comes from both substantially less work
and better use of the available P cores. Every timed invocation produced the
same semantic-result-prefix hash.

The individual changes were also checked with controlled A/Bs; selected
results below are not additive:

• contiguous predicted-row storage: 4.3% faster at -j1 and 4.1% at -j12;
• execution-state fresh-ID reservations: 14.8% faster at -j12;
• sorted-run column-index construction: 3.5% end-to-end at -j12, with the
  auxiliary-index rebuild itself improving from 278ms to 164ms;
• coarse rebuild partitions: 4.2% end-to-end at -j12;
• batched/row-aware deletion: 3.8% end-to-end at -j12, and 11–13% on the
  large deletion-only benchmark;
• concurrent union-find plus coarse join partitioning: 32.7% faster at -j12,
  with 22.5% less CPU and essentially neutral -j1; and
• the final packed-hash/no-PRB layout: a further 5.15% at -j1 and 5.60% at
  -j12.

The final math stage curve is search/apply 8.90x, mutation merge 8.20x, and
equality rebuild 5.45x from one to 12 threads. Rebuild remains the limiting
stage at roughly 65% of clean -j12 wall time.

Broad-suite safety check for the final layout

There is no cumulative c7fc1d77-to-tip sweep for the full 12-workload suite.
The broad sweep isolates the final packed-hash/no-PRB change against commit 11:

• eggcc-2mm is 2.35% faster at -j12, with 2.2% less CPU.
• The equal-weight new/baseline wall-time geomean is effectively parity
  (0.996x at -j1, 0.995x at -j12; both confidence intervals include
  1.0).
• Runtime-weighted totals improve by 1.51% at -j1 and 0.69% at -j12.

Validation

• cargo nextest run --release --all — 1,238 tests passed
• cargo test -p egglog-core-relations — 144 tests plus two doctests passed
• cargo clippy -p egglog-core-relations --all-targets -- -D warnings
• cargo fmt --all -- --check
• wasm32 example compilation
• All non-math benchmark outputs were byte-identical. Math's semantic output
  was identical; only timing/order-dependent print-stats diagnostics varied
  across parallel runs.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 89.21221% with 152 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.21%. Comparing base (d7ea2c3) to head (2f046f1).

Files with missing lines	Patch %	Lines
core-relations/src/hash_index/mod.rs	85.77%	34 Missing ⚠️
core-relations/src/table/rebuild.rs	77.06%	25 Missing ⚠️
core-relations/src/action/mod.rs	79.83%	24 Missing ⚠️
core-relations/src/free_join/execute.rs	92.00%	18 Missing ⚠️
core-relations/src/uf/mod.rs	88.23%	16 Missing ⚠️
core-relations/src/table/mod.rs	95.61%	15 Missing ⚠️
union-find/src/concurrent/atomic_int.rs	33.33%	12 Missing ⚠️
union-find/src/concurrent/uf.rs	96.34%	3 Missing ⚠️
core-relations/src/parallel.rs	95.12%	2 Missing ⚠️
union-find/src/lib.rs	91.30%	2 Missing ⚠️
... and 1 more

Additional details and impacted files

@@                           Coverage Diff                           @@
##           codex/demand-driven-join-scheduling     #962      +/-   ##
=======================================================================
+ Coverage                                87.02%   87.21%   +0.19%     
=======================================================================
  Files                                       96       96              
  Lines                                    33261    34185     +924     
=======================================================================
+ Hits                                     28945    29816     +871     
- Misses                                    4316     4369      +53     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[codspeed-hq]

Merging this PR will improve performance by 9.99%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 6 improved benchmarks
✅ 31 untouched benchmarks
⏩ 227 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	rust_rule_insert_loop[ops1000_funcs0]	628.6 µs	564 µs	+11.45%
⚡	Simulation	rust_rule_insert_loop[ops1000_funcs2000]	908.8 µs	820.2 µs	+10.81%
⚡	Simulation	rust_rule_insert_loop[ops1000_funcs200]	667.6 µs	604.4 µs	+10.46%
⚡	Simulation	rust_rule_insert_loop[ops100000_funcs2000]	49.2 ms	44.8 ms	+9.76%
⚡	Simulation	rust_rule_insert_loop[ops100000_funcs200]	49.6 ms	45.6 ms	+8.77%
⚡	Simulation	rust_rule_insert_loop[ops100000_funcs0]	49.6 ms	45.6 ms	+8.72%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ezr-better-rebuild (2f046f1) with codex/demand-driven-join-scheduling (d7ea2c3)}

Footnotes

1. 227 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩