Skip to content

Plan queries without tree decomposition by default - #46

Draft
oflatt-claude wants to merge 3 commits into
saulshanabrook:mainfrom
oflatt-claude:bench-no-decomp
Draft

Plan queries without tree decomposition by default#46
oflatt-claude wants to merge 3 commits into
saulshanabrook:mainfrom
oflatt-claude:bench-no-decomp

Conversation

@oflatt-claude

@oflatt-claude oflatt-claude commented Aug 1, 2026 •
edited
Loading

Copy link
Copy Markdown

Standalone, on top of main.

Why

Tree decomposition splits a query into bags and materializes each bag's boundary. On this suite the boundary is routinely wider than the split saves, so it costs more than it returns. It is worst under the term/proof encoding, whose view rows carry a proof column that becomes payload in every bag — but this is not an encoding problem, and the largest single win is on native.

Measured

bench.py, 6 rounds per endpoint/file, serial, this branch against main. Wall time, 95% CI, candidate/baseline so lower is better. Full reports with phase and ruleset detail are in the comments.

treatment suite total
native (off) 0.805–0.814x
term 0.852–0.871x
proofs 0.909–0.953x

Per file, native:

file baseline candidate ratio
luminal-llama 1.11 s 555–574 ms 0.500–0.517x
herbie 127–141 ms 103–118 ms 0.756–0.897x
eggcc-2mm-pass1 2.70–2.71 s 2.27–2.29 s 0.839–0.845x
hardboiled_conv1d_32 316–327 ms 288–303 ms 0.892–0.946x
pointer-analysis-small 9.04–19.1 ms 8.77–18.4 ms 0.568–1.64x (CI includes 1)
math-microbenchmark 1.04–1.06 s 1.04–1.07 s 0.991–1.02x (CI includes 1)

Nothing regresses, and peak RSS is unchanged everywhere. The win is attributable: on eggcc-2mm-pass1, Search goes 1.64 s to 1.26 s, which is +89.4% of the wall-time change. math-microbenchmark does not move because its Search is flat and Rebuild is 36% of its wall.

This makes proof overhead worse, and that is the project's headline metric

All three treatments get faster, but by different amounts — native 19%, term 14%, proofs 7% — so proof-mode wall time relative to the non-proof baseline rises by roughly 15%. Against the README's "under 2x the non-proof baseline" target, this is a regression.

That is not specific to this change. Any optimization that helps the backend generically will widen the ratio, because native is the denominator and has less absolute work to hide a fixed saving in. On luminal-llama the saving is largely per-rule query planning, which tree decomposition performs at rule-build time: about 534 ms for native and 383 ms for term, since the encoding's several thousand extra rules are trivial 2-3 atom maintenance rules that decompose instantly while the complex user rules are common to both. A near-fixed 500 ms is half of native's 1.1 s and a rounding error against proof mode's 7 s.

Lowering the ratio needs work on encoding-specific cost instead. The phase data says where: after this change, luminal-llama in term mode is 2.87 s wall of which 2.59 s (90%) is outside all recorded rulesets, and in proof mode 6.4 s of 7.1 s. Search is never more than 20% of proof-mode wall on any file in the suite.

Landing this is still worth it on absolute time; it just should not be read as progress toward the 2x target.

Interface

--decomp turns decomposition back on. The per-rule :no-decomp label still forces it off for one rule even when --decomp is set.

--no-decomp is kept, accepted and ignored, rather than removed. That is what lets one command line drive both this build and an older one that decomposes unless told not to, so a benchmark run against an older ref measures both sides in the same configuration. bench.py passes it for exactly that reason, and the preflight requires it so a target predating the flag fails with the existing capability error rather than an unknown-argument error mid-collection.

One snapshot moves

egglog-experimental's proofs/test_fresh_proof_testing records a different derivation: same rule, same substitution, same premises, reordered, with the let numbering following. Which match is found first depends on the plan, so this is the expected kind of movement. Proof-testing verifies the proof it prints, and the new output is byte-identical across three consecutive runs.

No shared (print-size) snapshot moves, so no e-graph content changes.

Context

This came out of investigating why the term/proof encoding's user-rule search is slower than native's. Decomposition is where most of that gap lives. Search time in user rulesets on luminal-llama, decomposition on then off: native 32 ms to 17 ms, term 1384 ms to 234 ms, proofs 1835 ms to 633 ms. So the encoding's user-rule search improves 5.9x and goes from 43x native to about 14x — a large gain, but not parity, and closing the rest is separate work. Search in the encoding's own maintenance rulesets is unchanged by this (10 ms either way on that file), which is why math-microbenchmark barely moves: its search cost is maintenance (1406 ms), not user rules (194 ms). The heuristic is also unstable: small semantics-preserving changes to a query swing per-file time by tens of percent in either direction, which makes it hard to measure anything else while it is on. Improving the cost model so decomposition pays where it should is separate work.

State
cargo test --release --workspace 1467 passed, 0 failed
-p egglog --test files 792 passed
-p egglog --lib 68 passed
clippy --all-targets, cargo fmt --check clean
uv run pytest tests/ 171 passed, 1 failed
make python-nits clean

The Python failure is test_profile_main_reports_filesystem_errors, which fails identically on unmodified main in this environment and is unrelated to this change.

🤖 Generated with Claude Code

@oflatt @claude
Benchmark without tree decomposition
860a747 
Tree decomposition splits a query into bags and materializes each bag's
boundary. On this suite the boundary is routinely wider than the split saves,
so it costs more than it returns -- most of all under the term/proof encoding,
whose view rows carry a proof column into every bag.

Measured on this commit, 3-4 rounds, serial, ratios are no-decomp/decomp:

                        native    term  proofs
  luminal-llama          0.465   0.625   0.783
  pointer-analysis       0.562   1.022   0.987
  eggcc-2mm-pass1        0.844   0.941   0.965
  herbie                 0.814   0.885   0.931
  hardboiled_conv1d_32   0.924   0.826   0.910
  math-microbenchmark    1.006   1.009   1.005
  geomean                0.742   0.873   0.927

Nothing regresses: the two ratios above 1 are 1.022 on a 45 ms file and 1.009
on math-microbenchmark, both inside run-to-run noise.

This only changes what the harness measures. egglog's own default is
untouched, so `--no-decomp` stays opt-in for every other caller, and the
preflight now requires the flag so a target that predates it fails with the
capability error rather than an unknown-argument one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 1, 2026 •
edited
Loading

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The change adds --no-decomp to preflight requirements and workload commands. Tests update expected flags for standard, DD, and proof-extraction modes.

Changes

Benchmarking tree decomposition control

Layer / File(s) Summary
Preflight capability requirements
benchmarking/collection.py, tests/test_collection.py		 Preflight defaults and collection checks require --no-decomp with --timing-summary. Tests verify standard and proof-extraction requirements.
Workload command construction and validation
benchmarking/targets.py, tests/test_workloads.py		 Workload commands include --no-decomp in main, DD, and proof-extraction modes. The fact-directory assertion uses the updated argument position.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested reviewers: saulshanabrook

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: benchmark queries run without tree decomposition by default.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codspeed-hq

codspeed-hq Bot commented Aug 1, 2026 •
edited
Loading

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 6 untouched benchmarks

Comparing oflatt-claude:bench-no-decomp (ff0fda3) with main (f72b0dd)

Open in CodSpeed

@oflatt @claude
Plan queries without tree decomposition by default
9ae3390 
Tree decomposition splits a query into bags and materializes each bag's
boundary. On this suite the boundary is routinely wider than the split saves,
so it costs more than it returns -- most of all under the term/proof encoding,
whose view rows carry a proof column that becomes payload in every bag. The
largest single win, though, is on the native backend.

`--decomp` turns it back on. `--no-decomp` is still accepted and ignored, so
one command line can drive this build and an older one that decomposes unless
told not to -- which is what keeps a benchmark run against an older ref
measuring both sides the same way.

One proof snapshot moves. `test_fresh_proof_testing` records a different but
equally valid derivation, because which match is found first depends on the
plan; proof-testing verifies the proof it prints, and the new output is
identical across three runs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@oflatt-claude oflatt-claude changed the title Benchmark without tree decomposition Plan queries without tree decomposition by default Aug 1, 2026
@oflatt-claude

oflatt-claude commented Aug 1, 2026

Copy link
Copy Markdown
Author

bench.py reports

Replacing the hand-made table in the description. Both at 6 rounds per endpoint/file, serial, --detail rulesets.

A — native, this branch vs main (the change this PR makes)

Summary — no-decomp main/off vs f72b0dd main/off

Metric Scope File(s) Ratio (95% CI) Result
Wall time Suite total 6 files 0.805–0.814x faster
Wall time Lowest-ratio file luminal-llama.egg 0.500–0.517x faster
Wall time Highest-ratio file math-microbenchmark.egg 0.991–1.02x CI includes 1
Peak RSS Lowest-ratio file math-microbenchmark.egg 0.975–1.00x CI includes 1
Peak RSS Highest-ratio file herbie.egg 1.00–1.00x CI includes 1

Ratios are candidate / baseline; below 1 is lower and above 1 is higher.

Wall time

File Baseline (95% CI) Candidate (95% CI) Ratio (95% CI) Result
math-microbenchmark.egg 1.04–1.06 s 1.04–1.07 s 0.991–1.02x CI includes 1
eggcc-2mm-pass1.egg 2.70–2.71 s 2.27–2.29 s 0.839–0.845x faster
pointer-analysis-small.egg 9.04–19.1 ms 8.77–18.4 ms 0.568–1.64x CI includes 1
hardboiled_conv1d_32.egg 316–327 ms 288–303 ms 0.892–0.946x faster
luminal-llama.egg 1.11–1.11 s 555–574 ms 0.500–0.517x faster
herbie.egg 127–141 ms 103–118 ms 0.756–0.897x faster

Peak RSS

The win is Search. On eggcc-2mm-pass1, Search goes 1.64–1.65 s to 1.26–1.27 s, which is +89.4% of the wall-time change. On math-microbenchmark Search is flat and Rebuild is 36% of wall, which is why that file does not move.

Full report A (phases and rulesets)

Benchmark Report

Comparison

Role Target Git Backend Treatment
Baseline f72b0dd f72b0dd main off
Candidate no-decomp ff0fda3 main off

6 file(s): math-microbenchmark.egg, eggcc-2mm-pass1.egg, pointer-analysis-small.egg (facts: /home/oflatt/egglog-encoding-ufdelete/.claude/worktrees/uf-clear/benchmarks/data/pointer-analysis-small), hardboiled_conv1d_32.egg, luminal-llama.egg, herbie.egg · 6 round(s) per endpoint/file · 300 s timeout per run · Report: /home/oflatt/.claude/jobs/ad5ae266/tmp/pr46.jsonl

Summary — no-decomp main/off vs f72b0dd main/off

Metric Scope File(s) Ratio (95% CI) Result
Wall time Suite total 6 files 0.805–0.814x faster
Wall time Lowest-ratio file luminal-llama.egg 0.500–0.517x faster
Wall time Highest-ratio file math-microbenchmark.egg 0.991–1.02x CI includes 1
Peak RSS Lowest-ratio file math-microbenchmark.egg 0.975–1.00x CI includes 1
Peak RSS Highest-ratio file herbie.egg 1.00–1.00x CI includes 1

Ratios are candidate / baseline; below 1 is lower and above 1 is higher.

Per-file results

Wall time

File Baseline (95% CI) Candidate (95% CI) Ratio (95% CI) Result
math-microbenchmark.egg 1.04–1.06 s 1.04–1.07 s 0.991–1.02x CI includes 1
eggcc-2mm-pass1.egg 2.70–2.71 s 2.27–2.29 s 0.839–0.845x faster
pointer-analysis-small.egg 9.04–19.1 ms 8.77–18.4 ms 0.568–1.64x CI includes 1
hardboiled_conv1d_32.egg 316–327 ms 288–303 ms 0.892–0.946x faster
luminal-llama.egg 1.11–1.11 s 555–574 ms 0.500–0.517x faster
herbie.egg 127–141 ms 103–118 ms 0.756–0.897x faster

Peak RSS

File Baseline (95% CI) Candidate (95% CI) Ratio (95% CI) Result
math-microbenchmark.egg 193.4–197.1 MiB 191.2–194.8 MiB 0.975–1.00x CI includes 1
eggcc-2mm-pass1.egg 103.0–103.0 MiB 103.0–103.0 MiB 1.00–1.00x CI includes 1
pointer-analysis-small.egg 103.0–103.0 MiB 103.0–103.0 MiB 1.00–1.00x CI includes 1
hardboiled_conv1d_32.egg 103.0–103.0 MiB 103.0–103.0 MiB 1.00–1.00x CI includes 1
luminal-llama.egg 103.0–103.0 MiB 103.0–103.0 MiB 1.00–1.00x CI includes 1
herbie.egg 103.0–103.0 MiB 103.0–103.0 MiB 1.00–1.00x CI includes 1

Phase comparison

Endpoint cells show a 95% CI (or one-round point) and that phase's share of endpoint wall time. Delta is the signed candidate − baseline mean; Δ contribution is the phase's share of the wall-time change and may be negative or exceed 100% when phases offset. Execution overhead is stored per-ruleset unattributed time. Outside recorded rulesets is wall time minus all five recorded phases; ! marks a negative residual.

Phase comparison — math-microbenchmark.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 210–214 ms · 20.2% 209–216 ms · 20.2% +945 us +20.0%
Apply 245–248 ms · 23.5% 248–252 ms · 23.7% +3.44 ms +72.9%
Execution overhead 1.44–1.68 ms · 0.148% 1.39–1.69 ms · 0.146% -17.1 us -0.362%
Merge 162–167 ms · 15.7% 162–169 ms · 15.7% +1.10 ms +23.2%
Rebuild 380–385 ms · 36.4% 380–384 ms · 36.2% -302 us -6.39%
Outside recorded rulesets 39.5–46.8 ms · 4.11% 39.2–46.2 ms · 4.05% -441 us -9.32%

Phase comparison — eggcc-2mm-pass1.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 1.64–1.65 s · 60.8% 1.26–1.27 s · 55.5% -382 ms +89.4%
Apply 44.6–45.1 ms · 1.66% 43.7–45.4 ms · 1.95% -301 us +0.0705%
Execution overhead 39.4–39.6 ms · 1.46% 26.8–27.2 ms · 1.18% -12.5 ms +2.93%
Merge 29.2–29.8 ms · 1.09% 30.1–31.4 ms · 1.35% +1.29 ms -0.302%
Rebuild 579–582 ms · 21.4% 576–583 ms · 25.4% -1.26 ms +0.294%
Outside recorded rulesets 360–372 ms · 13.5% 327–340 ms · 14.6% -32.3 ms +7.56%

Phase comparison — pointer-analysis-small.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 173–337 us · 1.81% 268–513 us · 2.88% +135 us -26.8%
Apply 231–461 us · 2.46% 233–494 us · 2.68% +17.4 us -3.46%
Execution overhead 33.6–76.2 us · 0.390% 36.2–66.9 us · 0.380% -3.30 us +0.655%
Merge 43.5–103 us · 0.519% 45.5–94.3 us · 0.515% -3.17 us +0.630%
Rebuild 100–229 us · 1.17% 103–212 us · 1.16% -7.09 us +1.41%
Outside recorded rulesets 8.46–17.9 ms · 93.6% 8.09–17.0 ms · 92.4% -643 us +128%

Phase comparison — hardboiled_conv1d_32.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Apply 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Execution overhead 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Merge 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 316–327 ms · 100% 288–303 ms · 100% -26.1 ms +100%

Phase comparison — luminal-llama.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 27.8–28.0 ms · 2.51% 16.4–16.6 ms · 2.92% -11.4 ms +2.09%
Apply 2.10–2.16 ms · 0.192% 2.17–2.21 ms · 0.387% +57.1 us -0.0105%
Execution overhead 1.37–1.40 ms · 0.125% 1.34–1.36 ms · 0.239% -33.9 us +0.00620%
Merge 1.81–2.09 ms · 0.176% 1.93–2.02 ms · 0.350% +24.7 us -0.00452%
Rebuild 6.10–6.28 ms · 0.557% 6.09–6.21 ms · 1.09% -37.2 us +0.00681%
Outside recorded rulesets 1.07–1.07 s · 96.4% 527–546 ms · 95.0% -534 ms +97.9%

Phase comparison — herbie.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 15.5–15.6 ms · 11.6% 15.2–15.4 ms · 13.8% -300 us +1.27%
Apply 8.12–8.43 ms · 6.17% 8.04–8.42 ms · 7.45% -48.7 us +0.206%
Execution overhead 1.68–1.72 ms · 1.27% 1.60–1.65 ms · 1.47% -75.6 us +0.321%
Merge 2.79–2.83 ms · 2.09% 2.71–2.81 ms · 2.50% -48.7 us +0.206%
Rebuild 4.62–4.75 ms · 3.50% 4.60–4.72 ms · 4.22% -23.4 us +0.0990%
Outside recorded rulesets 93.7–108 ms · 75.4% 70.5–85.2 ms · 70.5% -23.1 ms +97.9%

Ruleset comparison

Totals show a 95% CI or one-round point. S/A/Exec/M/R are signed candidate − baseline mean deltas for Search, Apply, Execution overhead (stored unattributed time), Merge, and Rebuild.

Ruleset comparison — math-microbenchmark.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
1.00–1.01 s 1.00–1.02 s +5.17 ms +945 us +3.44 ms -17.1 us +1.10 ms -302 us

Ruleset comparison — eggcc-2mm-pass1.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
type-analysis 545–548 ms 191–192 ms -355 ms -343 ms -210 us -11.7 ms +65.0 us -119 us
always-run 1.44–1.44 s 1.40–1.42 s -27.6 ms -27.7 ms +6.60 us -532 us +1.17 ms -545 us
passthrough 40.4–40.9 ms 32.7–33.6 ms -7.53 ms -7.11 ms -5.30 us -137 us -10.0 us -263 us
error-checking 25.9–26.1 ms 21.6–21.8 ms -4.30 ms -4.26 ms -1.04 us -36.3 us -120 ns 0 ns
loop-iters-analysis 963–972 us 4.38–4.47 ms +3.45 ms +3.44 ms -122 ns +17.5 us -153 ns 0 ns
state-edge-passthrough 10.2–10.5 ms 8.48–8.64 ms -1.79 ms -1.64 ms -1.59 us -7.75 us -3.16 us -143 us
boundary-analysis-prep 3.08–3.13 ms 1.85–1.91 ms -1.23 ms -1.22 ms +228 ns -11.0 us -2.45 us 0 ns
type-helpers 166–168 ms 166–168 ms -310 us -53.5 us -24.8 us +17.7 us +18.3 us -267 us
cicm-index 2.56–2.69 ms 2.36–2.43 ms -235 us -184 us -48.4 us -620 ns -1.38 us 0 ns
rec-to-loop 493–516 us 276–279 us -227 us -227 us 0 ns +585 ns -80.0 ns 0 ns

Showing 10 of 23 changed rulesets by absolute total delta.

Ruleset comparison — pointer-analysis-small.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
0.582–1.20 ms 0.686–1.38 ms +139 us +135 us +17.4 us -3.30 us -3.17 us -7.09 us

Ruleset comparison — hardboiled_conv1d_32.egg

No nonzero ruleset timing differences.

Ruleset comparison — luminal-llama.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
expr 27.2–27.7 ms 15.9–16.2 ms -11.4 ms -11.4 ms +3.80 us -21.0 us +47.6 us -37.2 us
base_cleanup 133–139 us 127–133 us -6.03 us -451 ns +496 ns -24.8 ns -6.05 us 0 ns
dtype_prop 6.56–6.71 ms 6.59–6.68 ms +3.41 us +30.9 us -5.81 us -12.0 us -9.61 us 0 ns
cuda_memory_analysis 866–877 us 852–887 us -2.59 us +1.95 us -2.07 us -1.45 us -1.02 us 0 ns
cleanup 4.44–4.51 ms 4.41–4.54 ms -2.19 us -57.4 us +60.7 us +670 ns -6.19 us 0 ns

Ruleset comparison — herbie.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
32.9–33.2 ms 32.3–32.8 ms -496 us -300 us -48.7 us -75.6 us -48.7 us -23.4 us

B — proof mode vs normal, both on this branch

Included because it shows where proof-mode time actually goes.

Summary — ff0fda3 main/proofs vs ff0fda3 main/off

Metric Scope File(s) Ratio (95% CI) Result
Wall time Suite total 6 files 7.39–7.59x slower
Wall time Lowest-ratio file hardboiled_conv1d_32.egg 4.52–4.76x slower
Wall time Highest-ratio file luminal-llama.egg 12.5–13.5x slower
Peak RSS Lowest-ratio file pointer-analysis-small.egg 1.00–1.00x CI includes 1
Peak RSS Highest-ratio file math-microbenchmark.egg 11.6–11.8x higher RSS

Ratios are candidate / baseline; below 1 is lower and above 1 is higher.

Search is never more than 20% of proof-mode wall time; the residual dominates:

file Search %wall Outside recorded rulesets %wall
math-microbenchmark 17.8% 3.67%
eggcc-2mm-pass1 19.9% 46.4%
hardboiled_conv1d_32 18.8% 60.4%
herbie 6.11% 70.3%
luminal-llama 8.99% 89.6%
pointer-analysis-small 1.75% 90.4%
Full report B (phases and rulesets)

Benchmark Report

Comparison

Role Target Git Backend Treatment
Baseline ff0fda3 ff0fda3 main off
Candidate ff0fda3 ff0fda3 main proofs

6 file(s): math-microbenchmark.egg, eggcc-2mm-pass1.egg, pointer-analysis-small.egg (facts: /home/oflatt/egglog-encoding-ufdelete/.claude/worktrees/uf-clear/benchmarks/data/pointer-analysis-small), hardboiled_conv1d_32.egg, luminal-llama.egg, herbie.egg · 6 round(s) per endpoint/file · 300 s timeout per run · Report: /home/oflatt/.claude/jobs/ad5ae266/tmp/pr46.jsonl

Summary — ff0fda3 main/proofs vs ff0fda3 main/off

Metric Scope File(s) Ratio (95% CI) Result
Wall time Suite total 6 files 7.39–7.59x slower
Wall time Lowest-ratio file hardboiled_conv1d_32.egg 4.52–4.76x slower
Wall time Highest-ratio file luminal-llama.egg 12.5–13.5x slower
Peak RSS Lowest-ratio file pointer-analysis-small.egg 1.00–1.00x CI includes 1
Peak RSS Highest-ratio file math-microbenchmark.egg 11.6–11.8x higher RSS

Ratios are candidate / baseline; below 1 is lower and above 1 is higher.

Per-file results

Wall time

File Baseline (95% CI) Candidate (95% CI) Ratio (95% CI) Result
math-microbenchmark.egg 1.04–1.07 s 9.68–10.2 s 9.16–9.68x slower
eggcc-2mm-pass1.egg 2.27–2.29 s 12.5–12.9 s 5.48–5.65x slower
pointer-analysis-small.egg 8.77–18.4 ms 67.2–81.6 ms 3.98–8.55x slower
hardboiled_conv1d_32.egg 288–303 ms 1.36–1.38 s 4.52–4.76x slower
luminal-llama.egg 555–574 ms 7.11–7.60 s 12.5–13.5x slower
herbie.egg 103–118 ms 946–967 ms 8.12–9.29x slower

Peak RSS

File Baseline (95% CI) Candidate (95% CI) Ratio (95% CI) Result
math-microbenchmark.egg 191.2–194.8 MiB 2.2–2.2 GiB 11.6–11.8x higher RSS
eggcc-2mm-pass1.egg 103.0–103.0 MiB 978.7–983.7 MiB 9.50–9.55x higher RSS
pointer-analysis-small.egg 103.0–103.0 MiB 103.0–103.0 MiB 1.00–1.00x CI includes 1
hardboiled_conv1d_32.egg 103.0–103.0 MiB 197.5–198.3 MiB 1.92–1.93x higher RSS
luminal-llama.egg 103.0–103.0 MiB 733.3–734.4 MiB 7.12–7.13x higher RSS
herbie.egg 103.0–103.0 MiB 125.2–126.4 MiB 1.22–1.23x higher RSS

Phase comparison

Endpoint cells show a 95% CI (or one-round point) and that phase's share of endpoint wall time. Delta is the signed candidate − baseline mean; Δ contribution is the phase's share of the wall-time change and may be negative or exceed 100% when phases offset. Execution overhead is stored per-ruleset unattributed time. Outside recorded rulesets is wall time minus all five recorded phases; ! marks a negative residual.

Phase comparison — math-microbenchmark.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 209–216 ms · 20.2% 1.69–1.84 s · 17.8% +1.55 s +17.5%
Apply 248–252 ms · 23.7% 2.83–2.91 s · 28.9% +2.62 s +29.5%
Execution overhead 1.39–1.69 ms · 0.146% 4.25–4.69 ms · 0.0450% +2.93 ms +0.0330%
Merge 162–169 ms · 15.7% 4.79–5.06 s · 49.6% +4.76 s +53.7%
Rebuild 380–384 ms · 36.2% 0–0 ns · 0% -382 ms -4.30%
Outside recorded rulesets 39.2–46.2 ms · 4.05% 349–380 ms · 3.67% +321 ms +3.62%

Phase comparison — eggcc-2mm-pass1.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 1.26–1.27 s · 55.5% 2.48–2.56 s · 19.9% +1.25 s +12.1%
Apply 43.7–45.4 ms · 1.95% 2.81–2.90 s · 22.5% +2.81 s +27.0%
Execution overhead 26.8–27.2 ms · 1.18% 84.8–90.4 ms · 0.691% +60.6 ms +0.583%
Merge 30.1–31.4 ms · 1.35% 1.31–1.35 s · 10.5% +1.30 s +12.5%
Rebuild 576–583 ms · 25.4% 0–0 ns · 0% -579 ms -5.57%
Outside recorded rulesets 327–340 ms · 14.6% 5.80–5.97 s · 46.4% +5.55 s +53.4%

Phase comparison — pointer-analysis-small.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 268–513 us · 2.88% 1.30–1.31 ms · 1.75% +913 us +1.50%
Apply 233–494 us · 2.68% 2.48–2.54 ms · 3.37% +2.15 ms +3.53%
Execution overhead 36.2–66.9 us · 0.380% 100–119 us · 0.147% +57.8 us +0.0950%
Merge 45.5–94.3 us · 0.515% 3.15–3.23 ms · 4.29% +3.12 ms +5.13%
Rebuild 103–212 us · 1.16% 0–0 ns · 0% -158 us -0.259%
Outside recorded rulesets 8.09–17.0 ms · 92.4% 60.1–74.5 ms · 90.4% +54.8 ms +90.0%

Phase comparison — hardboiled_conv1d_32.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 0–0 ns · 0% 256–258 ms · 18.8% +257 ms +23.9%
Apply 0–0 ns · 0% 130–133 ms · 9.59% +131 ms +12.2%
Execution overhead 0–0 ns · 0% 6.98–7.10 ms · 0.513% +7.04 ms +0.654%
Merge 0–0 ns · 0% 146–149 ms · 10.8% +147 ms +13.7%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 288–303 ms · 100% 817–838 ms · 60.4% +532 ms +49.5%

Phase comparison — luminal-llama.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 16.4–16.6 ms · 2.92% 644–678 ms · 8.99% +645 ms +9.49%
Apply 2.17–2.21 ms · 0.387% 27.8–30.0 ms · 0.393% +26.7 ms +0.394%
Execution overhead 1.34–1.36 ms · 0.239% 11.9–14.2 ms · 0.178% +11.7 ms +0.173%
Merge 1.93–2.02 ms · 0.350% 60.0–65.5 ms · 0.853% +60.8 ms +0.895%
Rebuild 6.09–6.21 ms · 1.09% 0–0 ns · 0% -6.15 ms -0.0906%
Outside recorded rulesets 527–546 ms · 95.0% 6.36–6.82 s · 89.6% +6.05 s +89.1%

Phase comparison — herbie.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 15.2–15.4 ms · 13.8% 58.0–59.0 ms · 6.11% +43.2 ms +5.11%
Apply 8.04–8.42 ms · 7.45% 113–116 ms · 12.0% +106 ms +12.5%
Execution overhead 1.60–1.65 ms · 1.47% 3.23–3.29 ms · 0.341% +1.63 ms +0.193%
Merge 2.71–2.81 ms · 2.50% 106–110 ms · 11.3% +105 ms +12.4%
Rebuild 4.60–4.72 ms · 4.22% 0–0 ns · 0% -4.66 ms -0.551%
Outside recorded rulesets 70.5–85.2 ms · 70.5% 666–680 ms · 70.3% +595 ms +70.3%

Ruleset comparison

Totals show a 95% CI or one-round point. S/A/Exec/M/R are signed candidate − baseline mean deltas for Search, Apply, Execution overhead (stored unattributed time), Merge, and Rebuild.

Ruleset comparison — math-microbenchmark.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
1.00–1.02 s 6.84–7.08 s +5.95 s +17.7 ms +2.40 s +597 us +3.91 s -382 ms
@Rebuilding — 2.27–2.49 s +2.38 s +1.40 s +212 ms +2.14 ms +770 ms 0 ns
@parent — 216–231 ms +224 ms +135 ms +6.64 ms +195 us +82.0 ms 0 ns

Ruleset comparison — eggcc-2mm-pass1.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
@Rebuilding — 2.82–2.91 s +2.87 s +661 ms +2.03 s +44.1 ms +128 ms 0 ns
always-run 1.40–1.42 s 3.02–3.11 s +1.65 s +469 ms +577 ms +7.71 ms +982 ms -383 ms
type-analysis 191–192 ms 368–377 ms +181 ms +47.3 ms +64.4 ms +2.41 ms +70.5 ms -3.44 ms
type-helpers 166–168 ms 69.9–73.9 ms -95.1 ms +6.39 ms +24.9 ms +1.11 ms +29.5 ms -157 ms
is-resolved 43.4–43.7 ms 94.1–99.4 ms +53.1 ms +12.8 ms +16.7 ms +844 us +22.7 ms 0 ns
terms 22.6–23.0 ms 69.5–77.4 ms +50.6 ms +9.96 ms +21.7 ms +261 us +25.2 ms -6.53 ms
initialization 3.18–3.25 ms 46.8–48.2 ms +44.3 ms -145 ns +42.8 ms -37.4 us +1.53 ms 0 ns
@parent — 36.3–39.0 ms +37.6 ms +24.6 ms +1.79 ms +3.36 ms +7.83 ms 0 ns
terms-helpers 12.3–12.5 ms 45.8–50.4 ms +35.7 ms +3.19 ms +13.9 ms +288 us +19.9 ms -1.52 ms
passthrough 32.7–33.6 ms 19.7–20.8 ms -12.9 ms +3.20 ms +1.76 ms +40.7 us +3.00 ms -20.9 ms

Showing 10 of 26 changed rulesets by absolute total delta.

Ruleset comparison — pointer-analysis-small.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
0.686–1.38 ms 4.67–4.77 ms +3.69 ms -84.9 us +1.85 ms -29.3 us +2.11 ms -158 us
@Rebuilding — 1.87–1.91 ms +1.89 ms +801 us +260 us +77.5 us +755 us 0 ns
@parent — 491–511 us +501 us +197 us +38.9 us +9.66 us +255 us 0 ns

Ruleset comparison — hardboiled_conv1d_32.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
— 447–451 ms +449 ms +219 ms +110 ms +4.47 ms +116 ms 0 ns
@Rebuilding — 48.1–48.9 ms +48.5 ms +27.1 ms +9.28 ms +1.83 ms +10.3 ms 0 ns
typechecking — 32.5–33.4 ms +33.0 ms +6.36 ms +11.6 ms +425 us +14.6 ms 0 ns
@parent — 12.8–13.0 ms +12.9 ms +5.08 ms +854 us +307 us +6.67 ms 0 ns

Ruleset comparison — luminal-llama.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
matmul_backend — 332–349 ms +341 ms +333 ms +916 us +3.46 ms +2.95 ms 0 ns
fusion_pair — 101–106 ms +104 ms +93.1 ms +4.99 ms +1.06 ms +4.43 ms 0 ns
fusion_grow — 68.8–72.2 ms +70.5 ms +64.8 ms +2.14 ms +460 us +3.07 ms 0 ns
fusion_merge — 67.2–70.0 ms +68.6 ms +68.1 ms +79.5 us +198 us +277 us 0 ns
matmul_flatten — 37.5–39.5 ms +38.5 ms +38.1 ms 0 ns +348 us +3.37 us 0 ns
cleanup 4.41–4.54 ms 27.8–32.2 ms +25.5 ms +1.51 ms +7.70 ms +57.1 us +16.3 ms 0 ns
@Rebuilding — 18.4–23.2 ms +20.8 ms +11.3 ms +472 us +5.20 ms +3.77 ms 0 ns
expr 15.9–16.2 ms 30.1–32.7 ms +15.3 ms +8.25 ms +3.33 ms +239 us +9.66 ms -6.15 ms
direct_kernel — 13.3–13.9 ms +13.6 ms +11.7 ms +219 us +189 us +1.47 ms 0 ns
dtype_prop 6.59–6.68 ms 18.0–20.4 ms +12.6 ms +2.89 ms +1.65 ms +150 us +7.88 ms 0 ns

Showing 10 of 17 changed rulesets by absolute total delta.

Ruleset comparison — herbie.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
32.3–32.8 ms 201–207 ms +171 ms +2.52 ms +95.9 ms +238 us +77.3 ms -4.66 ms
@Rebuilding — 64.4–65.7 ms +65.1 ms +35.2 ms +9.12 ms +1.22 ms +19.6 ms 0 ns
@parent — 15.0–15.5 ms +15.3 ms +5.55 ms +1.08 ms +178 us +8.45 ms 0 ns

@oflatt-claude

oflatt-claude commented Aug 1, 2026

Copy link
Copy Markdown
Author

Term and proofs against main, completing the picture

Suite totals, branch vs main, 6 rounds per endpoint/file:

treatment wall time (95% CI)
native (off) 0.805–0.814x
term 0.852–0.871x
proofs 0.909–0.953x

Everything improves, but unevenly, and that unevenness is the important part — see the caveat now in the description.

The two modes also win from different phases. On luminal-llama:

native term
Search 27.8 ms to 16.4 ms (+2.09% of change) 1.36 s to 241 ms (+74.1%)
Outside recorded rulesets 1.07 s to 527 ms (+97.9%) 2.97 s to 2.59 s (+25.2%)

Native had only 28 ms of search to begin with, so its entire win is non-ruleset time — most likely per-rule query planning, which tree decomposition performs at rule-build time and which is not attributed to any ruleset. That is a roughly fixed cost for a given rule set, which is why it is about half of native's 1.1 s runtime and a small dent in proof mode's 7 s. (Attribution inferred from the phase accounting; planning time was not isolated directly.)

Full report — term vs main

Benchmark Report

Comparison

Role Target Git Backend Treatment
Baseline f72b0dd f72b0dd main term
Candidate no-decomp ff0fda3 main term

6 file(s): math-microbenchmark.egg, eggcc-2mm-pass1.egg, pointer-analysis-small.egg (facts: /home/oflatt/egglog-encoding-ufdelete/.claude/worktrees/uf-clear/benchmarks/data/pointer-analysis-small), hardboiled_conv1d_32.egg, luminal-llama.egg, herbie.egg · 6 round(s) per endpoint/file · 300 s timeout per run · Report: /home/oflatt/.claude/jobs/ad5ae266/tmp/pr46.jsonl

Summary — no-decomp main/term vs f72b0dd main/term

Metric Scope File(s) Ratio (95% CI) Result
Wall time Suite total 6 files 0.852–0.871x faster
Wall time Lowest-ratio file luminal-llama.egg 0.651–0.659x faster
Wall time Highest-ratio file math-microbenchmark.egg 0.956–1.01x CI includes 1
Peak RSS Lowest-ratio file luminal-llama.egg 0.962–0.965x lower RSS
Peak RSS Highest-ratio file herbie.egg 1.00–1.00x CI includes 1

Ratios are candidate / baseline; below 1 is lower and above 1 is higher.

Per-file results

Wall time

File Baseline (95% CI) Candidate (95% CI) Ratio (95% CI) Result
math-microbenchmark.egg 3.12–3.29 s 3.13–3.17 s 0.956–1.01x CI includes 1
eggcc-2mm-pass1.egg 7.60–7.89 s 7.17–7.22 s 0.912–0.947x faster
pointer-analysis-small.egg 38.3–52.7 ms 34.7–49.1 ms 0.725–1.16x CI includes 1
hardboiled_conv1d_32.egg 667–702 ms 571–582 ms 0.820–0.866x faster
luminal-llama.egg 4.38–4.41 s 2.86–2.89 s 0.651–0.659x faster
herbie.egg 320–336 ms 281–292 ms 0.847–0.900x faster

Peak RSS

File Baseline (95% CI) Candidate (95% CI) Ratio (95% CI) Result
math-microbenchmark.egg 456.0–461.9 MiB 455.4–460.0 MiB 0.989–1.01x CI includes 1
eggcc-2mm-pass1.egg 222.1–223.2 MiB 216.7–217.6 MiB 0.972–0.978x lower RSS
pointer-analysis-small.egg 103.0–103.0 MiB 103.0–103.0 MiB 1.00–1.00x CI includes 1
hardboiled_conv1d_32.egg 103.0–103.0 MiB 103.0–103.0 MiB 1.00–1.00x CI includes 1
luminal-llama.egg 315.5–316.4 MiB 304.3–304.8 MiB 0.962–0.965x lower RSS
herbie.egg 103.0–103.0 MiB 103.0–103.0 MiB 1.00–1.00x CI includes 1

Phase comparison

Endpoint cells show a 95% CI (or one-round point) and that phase's share of endpoint wall time. Delta is the signed candidate − baseline mean; Δ contribution is the phase's share of the wall-time change and may be negative or exceed 100% when phases offset. Execution overhead is stored per-ruleset unattributed time. Outside recorded rulesets is wall time minus all five recorded phases; ! marks a negative residual.

Phase comparison — math-microbenchmark.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 1.59–1.68 s · 51.1% 1.60–1.62 s · 51.2% -26.4 ms +47.6%
Apply 523–547 ms · 16.7% 525–531 ms · 16.8% -6.38 ms +11.5%
Execution overhead 3.30–3.74 ms · 0.110% 3.30–3.39 ms · 0.106% -180 us +0.325%
Merge 905–957 ms · 29.0% 905–919 ms · 28.9% -19.0 ms +34.2%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 90.6–107 ms · 3.08% 87.9–102 ms · 3.02% -3.57 ms +6.42%

Phase comparison — eggcc-2mm-pass1.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 2.75–2.85 s · 36.2% 2.33–2.34 s · 32.4% -469 ms +85.7%
Apply 1.88–1.96 s · 24.8% 1.92–1.94 s · 26.7% +3.33 ms -0.610%
Execution overhead 89.7–97.2 ms · 1.21% 67.5–68.1 ms · 0.942% -25.6 ms +4.69%
Merge 168–178 ms · 2.24% 168–171 ms · 2.35% -4.04 ms +0.739%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 2.70–2.80 s · 35.5% 2.69–2.71 s · 37.5% -51.7 ms +9.46%

Phase comparison — pointer-analysis-small.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 1.26–1.29 ms · 2.80% 1.35–1.49 ms · 3.39% +147 us -4.07%
Apply 545–564 us · 1.22% 543–573 us · 1.33% +3.33 us -0.0924%
Execution overhead 97.4–131 us · 0.251% 97.3–124 us · 0.264% -3.34 us +0.0927%
Merge 821–881 us · 1.87% 814–845 us · 1.98% -21.7 us +0.603%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 35.5–49.9 ms · 93.9% 31.8–46.1 ms · 93.0% -3.73 ms +103%

Phase comparison — hardboiled_conv1d_32.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 301–312 ms · 44.8% 228–230 ms · 39.7% -77.7 ms +72.0%
Apply 23.9–25.4 ms · 3.60% 24.7–25.1 ms · 4.32% +238 us -0.221%
Execution overhead 8.55–9.18 ms · 1.30% 6.45–6.56 ms · 1.13% -2.36 ms +2.19%
Merge 16.8–18.2 ms · 2.56% 16.8–17.1 ms · 2.94% -584 us +0.542%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 315–338 ms · 47.7% 293–305 ms · 51.9% -27.5 ms +25.5%

Phase comparison — luminal-llama.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 1.36–1.37 s · 31.1% 241–244 ms · 8.43% -1.12 s +74.1%
Apply 5.42–5.56 ms · 0.125% 5.52–5.87 ms · 0.198% +203 us -0.0134%
Execution overhead 22.6–22.8 ms · 0.517% 11.2–11.2 ms · 0.390% -11.5 ms +0.760%
Merge 14.5–15.9 ms · 0.346% 16.2–17.0 ms · 0.577% +1.41 ms -0.0927%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 2.97–3.00 s · 67.9% 2.59–2.62 s · 90.4% -383 ms +25.2%

Phase comparison — herbie.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 55.4–56.0 ms · 17.0% 55.4–55.7 ms · 19.4% -135 us +0.323%
Apply 22.3–22.7 ms · 6.85% 22.4–22.7 ms · 7.87% +64.7 us -0.155%
Execution overhead 3.06–3.10 ms · 0.938% 2.96–3.00 ms · 1.04% -94.8 us +0.227%
Merge 20.4–20.8 ms · 6.28% 20.3–20.6 ms · 7.13% -174 us +0.417%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 218–234 ms · 69.0% 179–190 ms · 64.5% -41.4 ms +99.2%

Ruleset comparison

Totals show a 95% CI or one-round point. S/A/Exec/M/R are signed candidate − baseline mean deltas for Search, Apply, Execution overhead (stored unattributed time), Merge, and Rebuild.

Ruleset comparison — math-microbenchmark.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
@Rebuilding 1.81–1.91 s 1.83–1.84 s -28.4 ms -21.3 ms +1.29 ms -56.0 us -8.30 ms 0 ns
1.06–1.12 s 1.06–1.07 s -21.2 ms -3.21 ms -7.70 ms -117 us -10.1 ms 0 ns
@parent 152–161 ms 153–155 ms -2.48 ms -1.93 ms +32.9 us -6.93 us -574 us 0 ns

Ruleset comparison — eggcc-2mm-pass1.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
type-analysis 636–661 ms 219–221 ms -428 ms -404 ms -601 us -22.8 ms -926 us 0 ns
always-run 1.57–1.63 s 1.55–1.56 s -43.7 ms -41.1 ms -443 us -932 us -1.30 ms 0 ns
passthrough 23.0–23.8 ms 13.8–14.1 ms -9.41 ms -9.18 ms -7.25 us -252 us +28.5 us 0 ns
error-checking 29.5–30.5 ms 23.4–23.7 ms -6.46 ms -6.38 ms -1.56 us -71.8 us -440 ns 0 ns
loop-iters-analysis 1.17–1.19 ms 4.94–5.03 ms +3.81 ms +3.79 ms -259 ns +13.3 us +818 ns 0 ns
state-edge-passthrough 9.26–9.64 ms 6.11–6.19 ms -3.30 ms -3.18 ms +2.28 us -123 us -1.05 us 0 ns
@Rebuilding 2.45–2.56 s 2.49–2.51 s -2.03 ms -4.57 ms +4.56 ms -1.12 ms -900 us 0 ns
boundary-analysis-prep 4.33–4.57 ms 2.80–2.90 ms -1.60 ms -1.59 ms +816 ns -12.0 us -1.59 us 0 ns
@parent 23.3–24.4 ms 22.9–23.2 ms -736 us -567 us -22.2 us -122 us -24.6 us 0 ns
is-resolved 51.8–54.0 ms 51.9–52.4 ms -704 us -402 us -39.5 us -41.6 us -221 us 0 ns

Showing 10 of 26 changed rulesets by absolute total delta.

Ruleset comparison — pointer-analysis-small.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
1.03–1.09 ms 1.12–1.25 ms +128 us +132 us +6.92 us +1.36 us -12.3 us 0 ns
@Rebuilding 1.35–1.39 ms 1.35–1.39 ms -3.39 us +11.6 us -2.40 us -4.59 us -8.02 us 0 ns
@parent 354–371 us 351–374 us -75.7 ns +2.71 us -1.20 us -120 ns -1.47 us 0 ns

Ruleset comparison — hardboiled_conv1d_32.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
298–310 ms 223–225 ms -79.8 ms -77.3 ms +400 us -2.29 ms -629 us 0 ns
@Rebuilding 35.7–36.8 ms 35.6–36.0 ms -464 us -204 us -143 us -60.6 us -56.8 us 0 ns
@parent 7.26–7.55 ms 7.21–7.31 ms -146 us -113 us +879 ns -8.01 us -26.0 us 0 ns
typechecking 9.84–10.4 ms 10.1–10.3 ms +28.5 us -68.2 us -19.8 us -11.4 us +128 us 0 ns

Ruleset comparison — luminal-llama.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
fusion_pair 499–503 ms 45.9–46.7 ms -455 ms -450 ms +157 us -4.80 ms +82.5 us 0 ns
fusion_grow 476–481 ms 25.0–25.3 ms -453 ms -451 ms -11.0 us -2.17 ms +14.0 us 0 ns
matmul_backend 243–245 ms 97.9–99.0 ms -146 ms -143 ms +29.0 us -2.96 ms +449 us 0 ns
fusion_merge 56.3–56.7 ms 22.4–22.6 ms -34.0 ms -32.9 ms -695 ns -1.07 ms -18.5 us 0 ns
expr 40.3–41.3 ms 18.9–19.5 ms -21.6 ms -21.3 ms -20.2 us -93.8 us -239 us 0 ns
direct_kernel 30.0–30.6 ms 9.44–9.54 ms -20.8 ms -21.4 ms -64.7 ns -129 us +710 us 0 ns
matmul_flatten 15.9–16.2 ms 14.2–14.4 ms -1.75 ms -1.62 ms 0 ns -128 us -445 ns 0 ns
kernel_lower 5.42–5.46 ms 4.28–4.33 ms -1.14 ms -1.01 ms -3.13 us -38.3 us -87.8 us 0 ns
buffer_reuse 403–422 us 671–684 us +265 us +261 us 0 ns +3.74 us -238 ns 0 ns
glumoe 3.71–3.94 ms 3.51–3.66 ms -240 us -735 us +3.25 us -21.2 us +514 us 0 ns

Showing 10 of 17 changed rulesets by absolute total delta.

Ruleset comparison — herbie.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
45.4–45.9 ms 44.9–45.3 ms -573 us -425 us +42.3 us -68.1 us -122 us 0 ns
@Rebuilding 46.5–47.1 ms 46.9–47.2 ms +235 us +264 us +29.3 us -28.8 us -29.8 us 0 ns
@parent 9.28–9.47 ms 9.31–9.44 ms -980 ns +26.2 us -6.94 us +2.15 us -22.4 us 0 ns
Full report — proofs vs main

Benchmark Report

Comparison

Role Target Git Backend Treatment
Baseline f72b0dd f72b0dd main proofs
Candidate no-decomp ff0fda3 main proofs

6 file(s): math-microbenchmark.egg, eggcc-2mm-pass1.egg, pointer-analysis-small.egg (facts: /home/oflatt/egglog-encoding-ufdelete/.claude/worktrees/uf-clear/benchmarks/data/pointer-analysis-small), hardboiled_conv1d_32.egg, luminal-llama.egg, herbie.egg · 6 round(s) per endpoint/file · 300 s timeout per run · Report: /home/oflatt/.claude/jobs/ad5ae266/tmp/pr46.jsonl

Summary — no-decomp main/proofs vs f72b0dd main/proofs

Metric Scope File(s) Ratio (95% CI) Result
Wall time Suite total 6 files 0.909–0.953x faster
Wall time Lowest-ratio file luminal-llama.egg 0.804–0.868x faster
Wall time Highest-ratio file pointer-analysis-small.egg 0.887–1.17x CI includes 1
Peak RSS Lowest-ratio file luminal-llama.egg 0.983–0.985x lower RSS
Peak RSS Highest-ratio file pointer-analysis-small.egg 1.01–1.01x higher RSS

Ratios are candidate / baseline; below 1 is lower and above 1 is higher.

Per-file results

Wall time

File Baseline (95% CI) Candidate (95% CI) Ratio (95% CI) Result
math-microbenchmark.egg 9.61–10.6 s 9.68–10.2 s 0.928–1.04x CI includes 1
eggcc-2mm-pass1.egg 12.9–13.8 s 12.5–12.9 s 0.918–0.988x faster
pointer-analysis-small.egg 65.8–80.2 ms 67.2–81.6 ms 0.887–1.17x CI includes 1
hardboiled_conv1d_32.egg 1.46–1.48 s 1.36–1.38 s 0.923–0.945x faster
luminal-llama.egg 8.65–8.95 s 7.11–7.60 s 0.804–0.868x faster
herbie.egg 0.987–1.00 s 946–967 ms 0.950–0.975x faster

Peak RSS

File Baseline (95% CI) Candidate (95% CI) Ratio (95% CI) Result
math-microbenchmark.egg 2.2–2.2 GiB 2.2–2.2 GiB 0.997–1.00x CI includes 1
eggcc-2mm-pass1.egg 980.2–991.9 MiB 978.7–983.7 MiB 0.989–1.00x CI includes 1
pointer-analysis-small.egg 102.0–102.0 MiB 103.0–103.0 MiB 1.01–1.01x higher RSS
hardboiled_conv1d_32.egg 200.7–201.3 MiB 197.5–198.3 MiB 0.982–0.987x lower RSS
luminal-llama.egg 745.3–746.0 MiB 733.3–734.4 MiB 0.983–0.985x lower RSS
herbie.egg 125.8–126.2 MiB 125.2–126.4 MiB 0.994–1.00x CI includes 1

Phase comparison

Endpoint cells show a 95% CI (or one-round point) and that phase's share of endpoint wall time. Delta is the signed candidate − baseline mean; Δ contribution is the phase's share of the wall-time change and may be negative or exceed 100% when phases offset. Execution overhead is stored per-ruleset unattributed time. Outside recorded rulesets is wall time minus all five recorded phases; ! marks a negative residual.

Phase comparison — math-microbenchmark.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 1.68–1.87 s · 17.5% 1.69–1.84 s · 17.8% -10.3 ms +5.29%
Apply 2.78–3.02 s · 28.6% 2.83–2.91 s · 28.9% -30.1 ms +15.5%
Execution overhead 4.29–5.00 ms · 0.0459% 4.25–4.69 ms · 0.0450% -173 us +0.0890%
Merge 4.73–5.42 s · 50.1% 4.79–5.06 s · 49.6% -144 ms +74.4%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 363–384 ms · 3.69% 349–380 ms · 3.67% -9.15 ms +4.71%

Phase comparison — eggcc-2mm-pass1.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 2.98–3.13 s · 22.9% 2.48–2.56 s · 19.9% -535 ms +83.7%
Apply 2.75–2.92 s · 21.3% 2.81–2.90 s · 22.5% +21.4 ms -3.35%
Execution overhead 106–115 ms · 0.832% 84.8–90.4 ms · 0.691% -23.2 ms +3.62%
Merge 1.31–1.37 s · 10.0% 1.31–1.35 s · 10.5% -4.72 ms +0.737%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 5.74–6.23 s · 44.9% 5.80–5.97 s · 46.4% -98.1 ms +15.3%

Phase comparison — pointer-analysis-small.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 1.20–1.23 ms · 1.67% 1.30–1.31 ms · 1.75% +87.1 us +6.33%
Apply 2.44–2.75 ms · 3.55% 2.48–2.54 ms · 3.37% -83.8 us -6.10%
Execution overhead 99.5–114 us · 0.147% 100–119 us · 0.147% +2.41 us +0.175%
Merge 3.09–3.41 ms · 4.45% 3.15–3.23 ms · 4.29% -58.6 us -4.26%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 58.7–73.0 ms · 90.2% 60.1–74.5 ms · 90.4% +1.43 ms +104%

Phase comparison — hardboiled_conv1d_32.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 323–326 ms · 22.1% 256–258 ms · 18.8% -67.3 ms +69.0%
Apply 130–132 ms · 8.93% 130–133 ms · 9.59% +327 us -0.335%
Execution overhead 9.06–9.27 ms · 0.625% 6.98–7.10 ms · 0.513% -2.13 ms +2.19%
Merge 147–151 ms · 10.2% 146–149 ms · 10.8% -1.78 ms +1.83%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 846–862 ms · 58.2% 817–838 ms · 60.4% -26.6 ms +27.3%

Phase comparison — luminal-llama.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 1.81–1.83 s · 20.6% 644–678 ms · 8.99% -1.16 s +79.9%
Apply 27.7–28.4 ms · 0.319% 27.8–30.0 ms · 0.393% +862 us -0.0596%
Execution overhead 23.8–25.2 ms · 0.279% 11.9–14.2 ms · 0.178% -11.4 ms +0.791%
Merge 58.0–59.0 ms · 0.665% 60.0–65.5 ms · 0.853% +4.24 ms -0.293%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 6.73–7.02 s · 78.1% 6.36–6.82 s · 89.6% -285 ms +19.7%

Phase comparison — herbie.egg

Phase Baseline (95% CI · wall) Candidate (95% CI · wall) Delta Δ contribution
Search 58.4–58.8 ms · 5.90% 58.0–59.0 ms · 6.11% -122 us +0.330%
Apply 113–115 ms · 11.5% 113–116 ms · 12.0% +342 us -0.924%
Execution overhead 3.33–3.41 ms · 0.339% 3.23–3.29 ms · 0.341% -109 us +0.294%
Merge 107–108 ms · 10.8% 106–110 ms · 11.3% +564 us -1.52%
Rebuild 0–0 ns · 0% 0–0 ns · 0% 0 ns 0%
Outside recorded rulesets 704–716 ms · 71.5% 666–680 ms · 70.3% -37.7 ms +102%

Ruleset comparison

Totals show a 95% CI or one-round point. S/A/Exec/M/R are signed candidate − baseline mean deltas for Search, Apply, Execution overhead (stored unattributed time), Merge, and Rebuild.

Ruleset comparison — math-microbenchmark.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
6.72–7.57 s 6.84–7.08 s -181 ms -5.06 ms -33.8 ms -149 us -142 ms 0 ns
@Rebuilding 2.25–2.52 s 2.27–2.49 s -2.65 ms -5.15 ms +3.48 ms -21.7 us -954 us 0 ns
@parent 216–234 ms 216–231 ms -966 us -60.7 us +193 us -2.13 us -1.10 ms 0 ns

Ruleset comparison — eggcc-2mm-pass1.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
type-analysis 870–886 ms 368–377 ms -506 ms -480 ms -1.35 ms -21.1 ms -2.94 ms 0 ns
always-run 3.01–3.19 s 3.02–3.11 s -32.4 ms -34.1 ms +3.20 ms -644 us -773 us 0 ns
@Rebuilding 2.76–2.94 s 2.82–2.91 s +17.7 ms -462 us +20.1 ms -939 us -1.02 ms 0 ns
passthrough 29.7–30.7 ms 19.7–20.8 ms -9.95 ms -9.73 ms -63.0 us -139 us -18.1 us 0 ns
error-checking 32.9–34.4 ms 26.8–27.8 ms -6.41 ms -6.36 ms -252 ns -49.7 us -343 ns 0 ns
state-edge-passthrough 10.5–11.0 ms 6.40–6.95 ms -4.10 ms -3.96 ms -7.71 us -98.8 us -36.1 us 0 ns
loop-iters-analysis 1.42–1.49 ms 5.35–5.54 ms +3.99 ms +3.95 ms +1.02 us +24.3 us +6.74 us 0 ns
boundary-analysis-prep 5.94–6.33 ms 3.78–4.36 ms -2.06 ms -2.07 ms -7.41 us -9.20 us +21.8 us 0 ns
boundary-analysis 5.11–5.67 ms 4.07–4.49 ms -1.11 ms -1.01 ms -20.2 us -12.4 us -66.4 us 0 ns
is-resolved 95.1–100 ms 94.1–99.4 ms -1.03 ms -544 us -66.2 us -14.1 us -405 us 0 ns

Showing 10 of 26 changed rulesets by absolute total delta.

Ruleset comparison — pointer-analysis-small.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
4.59–4.95 ms 4.67–4.77 ms -54.0 us +84.8 us -82.1 us -828 ns -55.9 us 0 ns
@parent 485–506 us 491–511 us +4.96 us +507 ns -1.11 us +320 ns +5.25 us 0 ns
@Rebuilding 1.87–1.92 ms 1.87–1.91 ms -3.98 us +1.74 us -643 ns +2.92 us -8.00 us 0 ns

Ruleset comparison — hardboiled_conv1d_32.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
515–522 ms 447–451 ms -69.9 ms -67.2 ms +548 us -2.05 ms -1.28 ms 0 ns
typechecking 32.9–34.0 ms 32.5–33.4 ms -513 us -62.8 us -154 us -15.3 us -281 us 0 ns
@Rebuilding 48.6–49.0 ms 48.1–48.9 ms -263 us +2.69 us -65.8 us -61.0 us -139 us 0 ns
@parent 13.0–13.1 ms 12.8–13.0 ms -153 us -58.2 us -1.16 us -7.60 us -85.6 us 0 ns

Ruleset comparison — luminal-llama.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
fusion_grow 747–757 ms 68.8–72.2 ms -682 ms -680 ms -43.6 us -2.24 ms +136 us 0 ns
fusion_pair 584–589 ms 101–106 ms -483 ms -478 ms +186 us -4.91 ms -112 us 0 ns
expr 128–129 ms 30.1–32.7 ms -97.1 ms -96.4 ms +123 us -1.25 ms +457 us 0 ns
matmul_backend 255–259 ms 332–349 ms +83.3 ms +85.1 ms +107 us -2.32 ms +414 us 0 ns
matmul_flatten 17.7–18.2 ms 37.5–39.5 ms +20.5 ms +20.6 ms 0 ns -109 us +200 ns 0 ns
direct_kernel 22.0–22.3 ms 13.3–13.9 ms -8.54 ms -9.28 ms +8.73 us -115 us +847 us 0 ns
dtype_prop 20.9–21.9 ms 18.0–20.4 ms -2.20 ms -2.31 ms +60.9 us +12.1 us +37.1 us 0 ns
@Rebuilding 18.2–19.6 ms 18.4–23.2 ms +1.87 ms +1.09 ms +48.2 us +559 us +176 us 0 ns
cleanup 27.5–29.1 ms 27.8–32.2 ms +1.70 ms +66.4 us +231 us +26.1 us +1.38 ms 0 ns
buffer_reuse 450–474 us 2.05–2.17 ms +1.65 ms +1.64 ms 0 ns +9.49 us +677 ns 0 ns

Showing 10 of 17 changed rulesets by absolute total delta.

Ruleset comparison — herbie.egg

Ruleset Baseline total Candidate total Total Δ S Δ A Δ Exec Δ M Δ R Δ
@Rebuilding 64.5–65.0 ms 64.4–65.7 ms +333 us +332 us +66.5 us -25.1 us -40.7 us 0 ns
202–205 ms 201–207 ms +332 us -461 us +273 us -78.2 us +598 us 0 ns
@parent 15.2–15.3 ms 15.0–15.5 ms +10.5 us +6.82 us +2.93 us -5.56 us +6.28 us 0 ns

@oflatt
oflatt marked this pull request as draft August 2, 2026 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@oflatt-claude @oflatt