Skip to content

Use a per-benchmarks baseline instead of the last fully succesful run#8332

Merged
joseph-isaacs merged 2 commits into
developfrom
adamg/benchmarks-baseline
Jun 10, 2026
Merged

Use a per-benchmarks baseline instead of the last fully succesful run#8332
joseph-isaacs merged 2 commits into
developfrom
adamg/benchmarks-baseline

Conversation

@AdamGS

@AdamGS AdamGS commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Currently - when one post-merge benchmarks fails for any reason, the baseline we use is the last fully successful run. This PR changes it to be the last successful run of that specific benchmark.

Tested this works by running it in such a state and visually verifying the results.

Signed-off-by: Adam Gutglick <adam@spiraldb.com>
@AdamGS AdamGS added the action/benchmark-sql Trigger SQL benchmarks to run on this PR label Jun 10, 2026
@github-actions github-actions Bot removed the action/benchmark-sql Trigger SQL benchmarks to run on this PR label Jun 10, 2026
@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done 6a4663d 1 Explore Profiling Data
Previous Runs (1)
Status Commit Job Attempt Link
🟢 Done d0b8881 1 Explore Profiling Data

Powered by Polar Signals Cloud

@codspeed-hq

codspeed-hq Bot commented Jun 10, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 26.95%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 189 improved benchmarks
❌ 4 regressed benchmarks
✅ 1333 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation bitwise_not_vortex_buffer_mut[128] 216.9 ns 275.3 ns -21.19%
Simulation bitwise_not_vortex_buffer_mut[1024] 278.6 ns 336.9 ns -17.31%
Simulation bitwise_not_vortex_buffer_mut[2048] 342.2 ns 400.6 ns -14.56%
WallTime cuda/bitpacked_u8/unpack/3bw[100M] 299.2 µs 348 µs -14.02%
Simulation compare[63] 360.5 µs 244.8 µs +47.23%
Simulation chunked_bool_canonical_into[(1000, 10)] 46.4 µs 31.7 µs +46.03%
Simulation compare[56] 327.6 µs 224.9 µs +45.63%
Simulation compare[62] 363.9 µs 250 µs +45.54%
Simulation compare[63] 371.2 µs 255.5 µs +45.27%
Simulation compare[60] 353.8 µs 243.7 µs +45.18%
Simulation compare[56] 332.4 µs 229.7 µs +44.69%
Simulation compare[62] 368.7 µs 254.9 µs +44.68%
Simulation compare[61] 362.7 µs 250.8 µs +44.64%
Simulation compare[60] 358.6 µs 248.5 µs +44.32%
Simulation compare[58] 347.3 µs 240.9 µs +44.16%
Simulation compare[59] 354.5 µs 246.3 µs +43.93%
Simulation compare[61] 367.5 µs 255.6 µs +43.82%
Simulation compare[57] 345.9 µs 241.4 µs +43.28%
Simulation compare[58] 352.2 µs 245.8 µs +43.27%
Simulation compare[59] 359.3 µs 251.1 µs +43.1%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing adamg/benchmarks-baseline (6a4663d) with develop (47d2041)

Open in CodSpeed

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Benchmarks: PolarSignals Profiling

Vortex (geomean): 0.969x ➖

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (0.969x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
polarsignals_q00/datafusion:vortex-file-compressed 119964983 118031058 1.02
polarsignals_q01/datafusion:vortex-file-compressed 253916343 280729622 0.90
polarsignals_q02/datafusion:vortex-file-compressed 24352664 25691388 0.95
polarsignals_q03/datafusion:vortex-file-compressed 264550863 291718449 0.91
polarsignals_q04/datafusion:vortex-file-compressed 10453653 10745085 0.97
polarsignals_q05/datafusion:vortex-file-compressed 17677719 16333280 1.08
polarsignals_q06/datafusion:vortex-file-compressed 21091890 21235455 0.99
polarsignals_q07/datafusion:vortex-file-compressed 14308406 14838086 0.96
polarsignals_q08/datafusion:vortex-file-compressed 392787146 418480768 0.94
polarsignals_q09/datafusion:vortex-file-compressed 12679574 12954403 0.98

No file size changes detected.

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=1 on NVME

Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -10.3%
Engines: DataFusion No clear signal (-12.4%, environment too noisy confidence) · DuckDB No clear signal (-8.1%, environment too noisy confidence)
Vortex (geomean): 0.815x ✅
Parquet (geomean): 0.907x ➖
Shifts: Parquet (control) -9.3% · Median polish -15.7%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (0.786x ✅, 22↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-file-compressed 🚀 53551753 68030863 0.79
tpch_q02/datafusion:vortex-file-compressed 🚀 23314587 33172807 0.70
tpch_q03/datafusion:vortex-file-compressed 🚀 31463896 45605641 0.69
tpch_q04/datafusion:vortex-file-compressed 🚀 19869961 27600794 0.72
tpch_q05/datafusion:vortex-file-compressed 🚀 47113881 68628173 0.69
tpch_q06/datafusion:vortex-file-compressed 🚀 9720865 11991883 0.81
tpch_q07/datafusion:vortex-file-compressed 🚀 54071335 66761913 0.81
tpch_q08/datafusion:vortex-file-compressed 🚀 39498188 48505163 0.81
tpch_q09/datafusion:vortex-file-compressed 🚀 53625504 66295890 0.81
tpch_q10/datafusion:vortex-file-compressed 🚀 33269682 40766100 0.82
tpch_q11/datafusion:vortex-file-compressed 🚀 16652648 20754406 0.80
tpch_q12/datafusion:vortex-file-compressed 🚀 23384995 28690436 0.82
tpch_q13/datafusion:vortex-file-compressed 🚀 27481400 32456659 0.85
tpch_q14/datafusion:vortex-file-compressed 🚀 15185926 19066040 0.80
tpch_q15/datafusion:vortex-file-compressed 🚀 22877208 28187542 0.81
tpch_q16/datafusion:vortex-file-compressed 🚀 20356684 25073227 0.81
tpch_q17/datafusion:vortex-file-compressed 🚀 67553263 86949255 0.78
tpch_q18/datafusion:vortex-file-compressed 🚀 76888188 103607969 0.74
tpch_q19/datafusion:vortex-file-compressed 🚀 17706495 21421707 0.83
tpch_q20/datafusion:vortex-file-compressed 🚀 31318445 38001070 0.82
tpch_q21/datafusion:vortex-file-compressed 🚀 72064406 89371101 0.81
tpch_q22/datafusion:vortex-file-compressed 🚀 12251795 15034662 0.81
datafusion / vortex-compact (0.821x ✅, 21↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-compact 🚀 59831023 70186075 0.85
tpch_q02/datafusion:vortex-compact 🚀 25922723 30559876 0.85
tpch_q03/datafusion:vortex-compact 🚀 33367490 40469329 0.82
tpch_q04/datafusion:vortex-compact 🚀 22566860 26914396 0.84
tpch_q05/datafusion:vortex-compact 🚀 48157892 66158063 0.73
tpch_q06/datafusion:vortex-compact 🚀 12508834 16565162 0.76
tpch_q07/datafusion:vortex-compact 🚀 57696102 80419911 0.72
tpch_q08/datafusion:vortex-compact 🚀 43642531 51636481 0.85
tpch_q09/datafusion:vortex-compact 🚀 55255742 66481730 0.83
tpch_q10/datafusion:vortex-compact 🚀 37075966 44185303 0.84
tpch_q11/datafusion:vortex-compact 🚀 17895457 22228872 0.81
tpch_q12/datafusion:vortex-compact 🚀 30291080 35011475 0.87
tpch_q13/datafusion:vortex-compact 🚀 33179673 37542916 0.88
tpch_q14/datafusion:vortex-compact 🚀 18453194 23593716 0.78
tpch_q15/datafusion:vortex-compact 🚀 30838754 36300737 0.85
tpch_q16/datafusion:vortex-compact 🚀 24678628 30756502 0.80
tpch_q17/datafusion:vortex-compact 🚀 71630936 83923198 0.85
tpch_q18/datafusion:vortex-compact 🚀 78983495 99460165 0.79
tpch_q19/datafusion:vortex-compact 40127467 44367662 0.90
tpch_q20/datafusion:vortex-compact 🚀 35201416 42810975 0.82
tpch_q21/datafusion:vortex-compact 🚀 77058080 96513918 0.80
tpch_q22/datafusion:vortex-compact 🚀 13808144 16238479 0.85
datafusion / parquet (0.898x ✅, 12↑ 2↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/datafusion:parquet 🚨 150506150 117783972 1.28
tpch_q02/datafusion:parquet 75386053 81581070 0.92
tpch_q03/datafusion:parquet 99672947 96738332 1.03
tpch_q04/datafusion:parquet 50411775 52239475 0.97
tpch_q05/datafusion:parquet 🚨 125665157 111969359 1.12
tpch_q06/datafusion:parquet 46438037 48480677 0.96
tpch_q07/datafusion:parquet 135372182 134895900 1.00
tpch_q08/datafusion:parquet 107691199 108557488 0.99
tpch_q09/datafusion:parquet 133835076 143296129 0.93
tpch_q10/datafusion:parquet 🚀 124056853 143217712 0.87
tpch_q11/datafusion:parquet 🚀 43904232 51025120 0.86
tpch_q12/datafusion:parquet 🚀 74807004 94658594 0.79
tpch_q13/datafusion:parquet 🚀 201385994 231760858 0.87
tpch_q14/datafusion:parquet 47429565 50725203 0.94
tpch_q15/datafusion:parquet 🚀 59709569 70925996 0.84
tpch_q16/datafusion:parquet 🚀 44664530 59354047 0.75
tpch_q17/datafusion:parquet 🚀 138018323 187353253 0.74
tpch_q18/datafusion:parquet 🚀 157991242 195796878 0.81
tpch_q19/datafusion:parquet 🚀 77561593 88079506 0.88
tpch_q20/datafusion:parquet 🚀 73711315 86497700 0.85
tpch_q21/datafusion:parquet 🚀 146789860 187007828 0.78
tpch_q22/datafusion:parquet 🚀 45004045 59842766 0.75
datafusion / arrow (0.755x ✅, 19↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/datafusion:arrow 🚀 56597179 82922835 0.68
tpch_q02/datafusion:arrow 🚀 16736199 20126554 0.83
tpch_q03/datafusion:arrow 🚀 31625277 48753757 0.65
tpch_q04/datafusion:arrow 🚀 26972536 41210687 0.65
tpch_q05/datafusion:arrow 🚀 60771542 75661293 0.80
tpch_q06/datafusion:arrow 🚀 24661585 30661471 0.80
tpch_q07/datafusion:arrow 🚀 106895547 143797726 0.74
tpch_q08/datafusion:arrow 🚀 41135221 64965070 0.63
tpch_q09/datafusion:arrow 🚀 62738095 104331752 0.60
tpch_q10/datafusion:arrow 🚀 47409209 70589657 0.67
tpch_q11/datafusion:arrow 🚀 9249259 11121661 0.83
tpch_q12/datafusion:arrow 🚀 48230401 72534176 0.66
tpch_q13/datafusion:arrow 🚀 45003518 51558503 0.87
tpch_q14/datafusion:arrow 🚀 22708536 38668343 0.59
tpch_q15/datafusion:arrow 🚀 47206813 69380480 0.68
tpch_q16/datafusion:arrow 🚀 15959573 20496277 0.78
tpch_q17/datafusion:arrow 🚀 65802153 106418955 0.62
tpch_q18/datafusion:arrow 🚀 145424228 164495152 0.88
tpch_q19/datafusion:arrow 🚀 66596477 86472851 0.77
tpch_q20/datafusion:arrow 61936346 56700463 1.09
tpch_q21/datafusion:arrow 229243484 231093447 0.99
tpch_q22/datafusion:arrow 15917613 15446745 1.03
duckdb / vortex-file-compressed (0.809x ✅, 20↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-file-compressed 32067715 33812566 0.95
tpch_q02/duckdb:vortex-file-compressed 🚀 24985283 29927162 0.83
tpch_q03/duckdb:vortex-file-compressed 🚀 33714581 39452841 0.85
tpch_q04/duckdb:vortex-file-compressed 🚀 29132074 34128519 0.85
tpch_q05/duckdb:vortex-file-compressed 🚀 38006774 46494779 0.82
tpch_q06/duckdb:vortex-file-compressed 🚀 7920467 9294049 0.85
tpch_q07/duckdb:vortex-file-compressed 🚀 35508139 41703699 0.85
tpch_q08/duckdb:vortex-file-compressed 38077342 42276158 0.90
tpch_q09/duckdb:vortex-file-compressed 🚀 59472872 69575813 0.85
tpch_q10/duckdb:vortex-file-compressed 🚀 41306389 50938556 0.81
tpch_q11/duckdb:vortex-file-compressed 🚀 14571113 20322260 0.72
tpch_q12/duckdb:vortex-file-compressed 🚀 21986218 27460738 0.80
tpch_q13/duckdb:vortex-file-compressed 🚀 41676560 52015352 0.80
tpch_q14/duckdb:vortex-file-compressed 🚀 22524942 29154758 0.77
tpch_q15/duckdb:vortex-file-compressed 🚀 17189726 23175085 0.74
tpch_q16/duckdb:vortex-file-compressed 🚀 28890270 39094006 0.74
tpch_q17/duckdb:vortex-file-compressed 🚀 24091235 32725764 0.74
tpch_q18/duckdb:vortex-file-compressed 🚀 54032487 64155692 0.84
tpch_q19/duckdb:vortex-file-compressed 🚀 30466518 39200337 0.78
tpch_q20/duckdb:vortex-file-compressed 🚀 32814923 40458630 0.81
tpch_q21/duckdb:vortex-file-compressed 🚀 104647423 139270950 0.75
tpch_q22/duckdb:vortex-file-compressed 🚀 17424874 22557389 0.77
duckdb / vortex-compact (0.845x ✅, 22↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-compact 🚀 38310649 43817915 0.87
tpch_q02/duckdb:vortex-compact 🚀 33921348 40650124 0.83
tpch_q03/duckdb:vortex-compact 🚀 35322743 41614710 0.85
tpch_q04/duckdb:vortex-compact 🚀 30908328 36590074 0.84
tpch_q05/duckdb:vortex-compact 🚀 41852009 50298063 0.83
tpch_q06/duckdb:vortex-compact 🚀 10891692 12651984 0.86
tpch_q07/duckdb:vortex-compact 🚀 41225696 52203970 0.79
tpch_q08/duckdb:vortex-compact 🚀 42092004 49525028 0.85
tpch_q09/duckdb:vortex-compact 🚀 66098247 82898788 0.80
tpch_q10/duckdb:vortex-compact 🚀 45976995 57905972 0.79
tpch_q11/duckdb:vortex-compact 🚀 18244768 20611510 0.89
tpch_q12/duckdb:vortex-compact 🚀 29929569 34749084 0.86
tpch_q13/duckdb:vortex-compact 🚀 46997180 52596223 0.89
tpch_q14/duckdb:vortex-compact 🚀 26763033 31454074 0.85
tpch_q15/duckdb:vortex-compact 🚀 20636325 23915053 0.86
tpch_q16/duckdb:vortex-compact 🚀 32742116 37740797 0.87
tpch_q17/duckdb:vortex-compact 🚀 29008248 34644840 0.84
tpch_q18/duckdb:vortex-compact 🚀 53162253 59634300 0.89
tpch_q19/duckdb:vortex-compact 🚀 34949660 41966314 0.83
tpch_q20/duckdb:vortex-compact 🚀 39287815 46848340 0.84
tpch_q21/duckdb:vortex-compact 🚀 107275024 133993834 0.80
tpch_q22/duckdb:vortex-compact 🚀 18843831 21820554 0.86
duckdb / parquet (0.916x ➖, 10↑ 2↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/duckdb:parquet 76895664 83281131 0.92
tpch_q02/duckdb:parquet 🚀 39989686 48724990 0.82
tpch_q03/duckdb:parquet 🚀 71795837 87305877 0.82
tpch_q04/duckdb:parquet 🚀 50285480 57008390 0.88
tpch_q05/duckdb:parquet 🚀 69230838 77753461 0.89
tpch_q06/duckdb:parquet 🚨 27286857 24700446 1.10
tpch_q07/duckdb:parquet 🚀 77465979 89493812 0.87
tpch_q08/duckdb:parquet 90036738 91556979 0.98
tpch_q09/duckdb:parquet 🚀 143584891 178286737 0.81
tpch_q10/duckdb:parquet 132201520 146416738 0.90
tpch_q11/duckdb:parquet 🚀 23247678 26005825 0.89
tpch_q12/duckdb:parquet 🚀 47533276 55000133 0.86
tpch_q13/duckdb:parquet 🚀 265685967 327377376 0.81
tpch_q14/duckdb:parquet 53522537 59236201 0.90
tpch_q15/duckdb:parquet 🚨 34948119 30863089 1.13
tpch_q16/duckdb:parquet 67994557 67018833 1.01
tpch_q17/duckdb:parquet 58342548 60879262 0.96
tpch_q18/duckdb:parquet 127645928 128183252 1.00
tpch_q19/duckdb:parquet 91207547 94116071 0.97
tpch_q20/duckdb:parquet 67179272 70284705 0.96
tpch_q21/duckdb:parquet 🚀 177392185 219104855 0.81
tpch_q22/duckdb:parquet 55434024 58884630 0.94
duckdb / duckdb (0.873x ✅, 14↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/duckdb:duckdb 17885755 19256932 0.93
tpch_q02/duckdb:duckdb 🚀 15383534 17622594 0.87
tpch_q03/duckdb:duckdb 🚀 23532579 26895446 0.87
tpch_q04/duckdb:duckdb 🚀 21393309 25696143 0.83
tpch_q05/duckdb:duckdb 23958389 25769477 0.93
tpch_q06/duckdb:duckdb 7310512 7737604 0.94
tpch_q07/duckdb:duckdb 🚀 26195859 34164166 0.77
tpch_q08/duckdb:duckdb 🚀 24617622 32278653 0.76
tpch_q09/duckdb:duckdb 🚀 59757404 78392492 0.76
tpch_q10/duckdb:duckdb 🚀 53233013 67139385 0.79
tpch_q11/duckdb:duckdb 🚀 7579782 9003097 0.84
tpch_q12/duckdb:duckdb 🚀 18128437 20767955 0.87
tpch_q13/duckdb:duckdb 🚀 40811210 46020744 0.89
tpch_q14/duckdb:duckdb 22442077 24764350 0.91
tpch_q15/duckdb:duckdb 13869004 15124444 0.92
tpch_q16/duckdb:duckdb 🚀 26707211 30237966 0.88
tpch_q17/duckdb:duckdb 16702158 18084730 0.92
tpch_q18/duckdb:duckdb 🚀 41040715 46176424 0.89
tpch_q19/duckdb:duckdb 🚀 32059874 35837027 0.89
tpch_q20/duckdb:duckdb 26243049 27706164 0.95
tpch_q21/duckdb:duckdb 🚀 63626687 71580865 0.89
tpch_q22/duckdb:duckdb 25924936 27901763 0.93

File Size Changes (10 files changed, +0.0% overall, 7↑ 3↓)
File Scale Format Base HEAD Change %
supplier_0.vortex 1.0 vortex-file-compressed 603.53 KB 613.66 KB +10.13 KB +1.7%
orders_0.vortex 1.0 vortex-compact 31.73 MB 32.20 MB +482.27 KB +1.5%
part_0.vortex 1.0 vortex-compact 3.37 MB 3.39 MB +22.38 KB +0.6%
customer_0.vortex 1.0 vortex-file-compressed 8.89 MB 8.89 MB +7.84 KB +0.1%
orders_0.vortex 1.0 vortex-file-compressed 38.32 MB 38.34 MB +16.62 KB +0.0%
lineitem_0.vortex 1.0 vortex-file-compressed 85.14 MB 85.17 MB +31.34 KB +0.0%
lineitem_1.vortex 1.0 vortex-file-compressed 84.54 MB 84.56 MB +23.73 KB +0.0%
partsupp_0.vortex 1.0 vortex-file-compressed 23.81 MB 23.80 MB 13.29 KB -0.1%
partsupp_0.vortex 1.0 vortex-compact 20.91 MB 20.71 MB 196.91 KB -0.9%
part_0.vortex 1.0 vortex-file-compressed 4.94 MB 4.68 MB 266.45 KB -5.3%

Totals:

  • vortex-compact: 190.23 MB → 190.53 MB (+0.2%)
  • vortex-file-compressed: 246.50 MB → 246.32 MB (-0.1%)

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Benchmarks: FineWeb NVMe

Verdict: No clear signal (low confidence)
Attributed Vortex impact: +1.9%
Engines: DataFusion No clear signal (+2.3%, low confidence) · DuckDB No clear signal (+1.5%, low confidence)
Vortex (geomean): 1.068x ➖
Parquet (geomean): 1.049x ➖
Shifts: Parquet (control) +4.9% · Median polish +5.0%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.107x ❌, 0↑ 3↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
fineweb_q00/datafusion:vortex-file-compressed 🚨 5893138 5046800 1.17
fineweb_q01/datafusion:vortex-file-compressed 34616690 32665854 1.06
fineweb_q02/datafusion:vortex-file-compressed 39310712 37981598 1.03
fineweb_q03/datafusion:vortex-file-compressed 🚨 70338414 56061883 1.25
fineweb_q04/datafusion:vortex-file-compressed 294396907 273244445 1.08
fineweb_q05/datafusion:vortex-file-compressed 234129056 216184890 1.08
fineweb_q06/datafusion:vortex-file-compressed 🚨 59904283 48475734 1.24
fineweb_q07/datafusion:vortex-file-compressed 57079838 55447016 1.03
fineweb_q08/datafusion:vortex-file-compressed 22595470 21552056 1.05
datafusion / vortex-compact (1.045x ➖, 0↑ 1↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
fineweb_q00/datafusion:vortex-compact 🚨 6197879 5599436 1.11
fineweb_q01/datafusion:vortex-compact 101061643 94463996 1.07
fineweb_q02/datafusion:vortex-compact 107192849 102021051 1.05
fineweb_q03/datafusion:vortex-compact 880126259 858754439 1.02
fineweb_q04/datafusion:vortex-compact 947035710 913888071 1.04
fineweb_q05/datafusion:vortex-compact 846784886 823929938 1.03
fineweb_q06/datafusion:vortex-compact 481325543 459657592 1.05
fineweb_q07/datafusion:vortex-compact 489115510 477760431 1.02
fineweb_q08/datafusion:vortex-compact 21005272 20594714 1.02
datafusion / parquet (1.052x ➖, 0↑ 1↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
fineweb_q00/datafusion:parquet 7401858 7507700 0.99
fineweb_q01/datafusion:parquet 306530013 287348276 1.07
fineweb_q02/datafusion:parquet 297573391 290152605 1.03
fineweb_q03/datafusion:parquet 🚨 311964936 273966975 1.14
fineweb_q04/datafusion:parquet 330824498 304351007 1.09
fineweb_q05/datafusion:parquet 312029782 296068233 1.05
fineweb_q06/datafusion:parquet 300019235 298060181 1.01
fineweb_q07/datafusion:parquet 291937784 278848840 1.05
fineweb_q08/datafusion:parquet 292193104 275324423 1.06
duckdb / vortex-file-compressed (1.077x ➖, 0↑ 3↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
fineweb_q00/duckdb:vortex-file-compressed 🚨 3921318 3400711 1.15
fineweb_q01/duckdb:vortex-file-compressed 35224460 33299780 1.06
fineweb_q02/duckdb:vortex-file-compressed 🚨 44210467 38364666 1.15
fineweb_q03/duckdb:vortex-file-compressed 167783311 161048803 1.04
fineweb_q04/duckdb:vortex-file-compressed 291022516 264799848 1.10
fineweb_q05/duckdb:vortex-file-compressed 235589559 221101010 1.07
fineweb_q06/duckdb:vortex-file-compressed 🚨 56035207 49369579 1.14
fineweb_q07/duckdb:vortex-file-compressed 57893244 52842441 1.10
fineweb_q08/duckdb:vortex-file-compressed 21366986 23294176 0.92
duckdb / vortex-compact (1.045x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
fineweb_q00/duckdb:vortex-compact 4329874 4060616 1.07
fineweb_q01/duckdb:vortex-compact 109087500 104758296 1.04
fineweb_q02/duckdb:vortex-compact 120769423 111874830 1.08
fineweb_q03/duckdb:vortex-compact 887329758 860201533 1.03
fineweb_q04/duckdb:vortex-compact 929944675 897279312 1.04
fineweb_q05/duckdb:vortex-compact 831601036 797464511 1.04
fineweb_q06/duckdb:vortex-compact 472074442 450330308 1.05
fineweb_q07/duckdb:vortex-compact 493490451 480357212 1.03
fineweb_q08/duckdb:vortex-compact 19128145 18486468 1.03
duckdb / parquet (1.046x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
fineweb_q00/duckdb:parquet 33223451 31702879 1.05
fineweb_q01/duckdb:parquet 86660815 84160151 1.03
fineweb_q02/duckdb:parquet 90967008 84348127 1.08
fineweb_q03/duckdb:parquet 329057394 316352067 1.04
fineweb_q04/duckdb:parquet 455741421 444113863 1.03
fineweb_q05/duckdb:parquet 424285401 413781452 1.03
fineweb_q06/duckdb:parquet 210166900 202785552 1.04
fineweb_q07/duckdb:parquet 223818453 214799020 1.04
fineweb_q08/duckdb:parquet 36107846 33183532 1.09

File Size Changes (1 files changed, -0.1% overall, 0↑ 1↓)
File Scale Format Base HEAD Change %
sample.vortex 1.0 vortex-file-compressed 1.43 GB 1.43 GB 2.53 MB -0.2%

Totals:

  • vortex-compact: 1.23 GB → 1.23 GB (0.0%)
  • vortex-file-compressed: 1.43 GB → 1.43 GB (-0.2%)

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-DS SF=1 on NVME

Verdict: No clear signal (low confidence)
Attributed Vortex impact: +0.3%
Engines: DataFusion No clear signal (-0.1%, low confidence) · DuckDB No clear signal (+0.6%, low confidence)
Vortex (geomean): 1.014x ➖
Parquet (geomean): 1.011x ➖
Shifts: Parquet (control) +1.1% · Median polish +1.1%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.017x ➖, 0↑ 1↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpcds_q01/datafusion:vortex-file-compressed 25098271 25450651 0.99
tpcds_q02/datafusion:vortex-file-compressed 48252671 48153862 1.00
tpcds_q03/datafusion:vortex-file-compressed 15309090 15230580 1.01
tpcds_q04/datafusion:vortex-file-compressed 258226310 248213387 1.04
tpcds_q05/datafusion:vortex-file-compressed 42794804 45233121 0.95
tpcds_q06/datafusion:vortex-file-compressed 23321142 22827658 1.02
tpcds_q07/datafusion:vortex-file-compressed 41722704 39829184 1.05
tpcds_q08/datafusion:vortex-file-compressed 29702058 29560640 1.00
tpcds_q09/datafusion:vortex-file-compressed 31266542 31349172 1.00
tpcds_q10/datafusion:vortex-file-compressed 39438123 37982196 1.04
tpcds_q11/datafusion:vortex-file-compressed 133316324 130310947 1.02
tpcds_q12/datafusion:vortex-file-compressed 19356137 18171830 1.07
tpcds_q13/datafusion:vortex-file-compressed 44555741 44602402 1.00
tpcds_q14/datafusion:vortex-file-compressed 162683008 166780880 0.98
tpcds_q15/datafusion:vortex-file-compressed 27035898 27045708 1.00
tpcds_q16/datafusion:vortex-file-compressed 25330319 23827670 1.06
tpcds_q17/datafusion:vortex-file-compressed 63332869 59001516 1.07
tpcds_q18/datafusion:vortex-file-compressed 64115510 62883779 1.02
tpcds_q19/datafusion:vortex-file-compressed 21866340 21942038 1.00
tpcds_q20/datafusion:vortex-file-compressed 20946033 21431775 0.98
tpcds_q21/datafusion:vortex-file-compressed 36626081 36058998 1.02
tpcds_q22/datafusion:vortex-file-compressed 124345385 119359277 1.04
tpcds_q23/datafusion:vortex-file-compressed 150518303 148980068 1.01
tpcds_q24/datafusion:vortex-file-compressed 83756622 82729169 1.01
tpcds_q25/datafusion:vortex-file-compressed 66728539 64159504 1.04
tpcds_q26/datafusion:vortex-file-compressed 33769925 33725550 1.00
tpcds_q27/datafusion:vortex-file-compressed 95174343 94381571 1.01
tpcds_q28/datafusion:vortex-file-compressed 29159733 30122327 0.97
tpcds_q29/datafusion:vortex-file-compressed 60036037 60113195 1.00
tpcds_q30/datafusion:vortex-file-compressed 24120866 23474687 1.03
tpcds_q31/datafusion:vortex-file-compressed 74788615 73725641 1.01
tpcds_q32/datafusion:vortex-file-compressed 18765740 18456425 1.02
tpcds_q33/datafusion:vortex-file-compressed 28856759 29663686 0.97
tpcds_q34/datafusion:vortex-file-compressed 24790025 24417259 1.02
tpcds_q35/datafusion:vortex-file-compressed 45071403 43866783 1.03
tpcds_q36/datafusion:vortex-file-compressed 58146315 57331186 1.01
tpcds_q37/datafusion:vortex-file-compressed 20766182 20323630 1.02
tpcds_q38/datafusion:vortex-file-compressed 43109267 43086605 1.00
tpcds_q39/datafusion:vortex-file-compressed 107557704 107471541 1.00
tpcds_q40/datafusion:vortex-file-compressed 29949050 30719408 0.97
tpcds_q41/datafusion:vortex-file-compressed 15665827 15398526 1.02
tpcds_q42/datafusion:vortex-file-compressed 13859351 13714582 1.01
tpcds_q43/datafusion:vortex-file-compressed 19565495 18814361 1.04
tpcds_q44/datafusion:vortex-file-compressed 30073476 28765238 1.05
tpcds_q45/datafusion:vortex-file-compressed 26948031 27476850 0.98
tpcds_q46/datafusion:vortex-file-compressed 34964219 37054855 0.94
tpcds_q47/datafusion:vortex-file-compressed 136329242 135260828 1.01
tpcds_q48/datafusion:vortex-file-compressed 33414022 33296490 1.00
tpcds_q49/datafusion:vortex-file-compressed 57997074 57326852 1.01
tpcds_q50/datafusion:vortex-file-compressed 39281280 37887411 1.04
tpcds_q51/datafusion:vortex-file-compressed 90334110 88408889 1.02
tpcds_q52/datafusion:vortex-file-compressed 14380555 14224220 1.01
tpcds_q53/datafusion:vortex-file-compressed 21952479 22081848 0.99
tpcds_q54/datafusion:vortex-file-compressed 34052018 33322269 1.02
tpcds_q55/datafusion:vortex-file-compressed 13977585 13120245 1.07
tpcds_q56/datafusion:vortex-file-compressed 30421060 30223983 1.01
tpcds_q57/datafusion:vortex-file-compressed 107588388 110173738 0.98
tpcds_q58/datafusion:vortex-file-compressed 56876666 57352370 0.99
tpcds_q59/datafusion:vortex-file-compressed 59911269 58551564 1.02
tpcds_q60/datafusion:vortex-file-compressed 29178875 28806294 1.01
tpcds_q61/datafusion:vortex-file-compressed 40885326 40893617 1.00
tpcds_q62/datafusion:vortex-file-compressed 24771291 23827784 1.04
tpcds_q63/datafusion:vortex-file-compressed 22933326 22102159 1.04
tpcds_q64/datafusion:vortex-file-compressed 384558827 376906814 1.02
tpcds_q65/datafusion:vortex-file-compressed 51794127 50151423 1.03
tpcds_q66/datafusion:vortex-file-compressed 71502990 70445459 1.02
tpcds_q67/datafusion:vortex-file-compressed 155191067 155450440 1.00
tpcds_q68/datafusion:vortex-file-compressed 33283189 32593749 1.02
tpcds_q69/datafusion:vortex-file-compressed 36432826 35477877 1.03
tpcds_q70/datafusion:vortex-file-compressed 94036559 93553121 1.01
tpcds_q71/datafusion:vortex-file-compressed 23912364 22082618 1.08
tpcds_q72/datafusion:vortex-file-compressed 2147734810 2108980072 1.02
tpcds_q73/datafusion:vortex-file-compressed 23511716 23976493 0.98
tpcds_q74/datafusion:vortex-file-compressed 82533842 81600480 1.01
tpcds_q75/datafusion:vortex-file-compressed 110907232 109331269 1.01
tpcds_q76/datafusion:vortex-file-compressed 32627588 31286149 1.04
tpcds_q77/datafusion:vortex-file-compressed 37674828 40510389 0.93
tpcds_q78/datafusion:vortex-file-compressed 115465265 114741663 1.01
tpcds_q79/datafusion:vortex-file-compressed 30518965 28437469 1.07
tpcds_q80/datafusion:vortex-file-compressed 87336644 86542191 1.01
tpcds_q81/datafusion:vortex-file-compressed 25479247 24471868 1.04
tpcds_q82/datafusion:vortex-file-compressed 23530808 22330804 1.05
tpcds_q83/datafusion:vortex-file-compressed 36304394 35375142 1.03
tpcds_q84/datafusion:vortex-file-compressed 13442475 12452093 1.08
tpcds_q85/datafusion:vortex-file-compressed 88441124 84255626 1.05
tpcds_q86/datafusion:vortex-file-compressed 17406204 16892382 1.03
tpcds_q87/datafusion:vortex-file-compressed 44165920 41947819 1.05
tpcds_q88/datafusion:vortex-file-compressed 57354850 54149137 1.06
tpcds_q89/datafusion:vortex-file-compressed 25198226 24806784 1.02
tpcds_q90/datafusion:vortex-file-compressed 14686159 13577662 1.08
tpcds_q91/datafusion:vortex-file-compressed 18181337 17766585 1.02
tpcds_q92/datafusion:vortex-file-compressed 17228042 16359821 1.05
tpcds_q93/datafusion:vortex-file-compressed 33020731 31602388 1.04
tpcds_q94/datafusion:vortex-file-compressed 21888328 21337980 1.03
tpcds_q95/datafusion:vortex-file-compressed 58292705 58113758 1.00
tpcds_q96/datafusion:vortex-file-compressed 12796965 13428914 0.95
tpcds_q97/datafusion:vortex-file-compressed 30348277 30746274 0.99
tpcds_q98/datafusion:vortex-file-compressed 23745768 24128666 0.98
tpcds_q99/datafusion:vortex-file-compressed 🚨 31310371 27650671 1.13
datafusion / vortex-compact (1.010x ➖, 2↑ 1↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpcds_q01/datafusion:vortex-compact 26295431 26035746 1.01
tpcds_q02/datafusion:vortex-compact 55439726 53125501 1.04
tpcds_q03/datafusion:vortex-compact 20155291 19353847 1.04
tpcds_q04/datafusion:vortex-compact 313366034 292203222 1.07
tpcds_q05/datafusion:vortex-compact 46711818 48198204 0.97
tpcds_q06/datafusion:vortex-compact 27380855 26494959 1.03
tpcds_q07/datafusion:vortex-compact 52105280 52420921 0.99
tpcds_q08/datafusion:vortex-compact 37283106 36237218 1.03
tpcds_q09/datafusion:vortex-compact 44460359 44897099 0.99
tpcds_q10/datafusion:vortex-compact 51278661 47984283 1.07
tpcds_q11/datafusion:vortex-compact 152119993 159885579 0.95
tpcds_q12/datafusion:vortex-compact 🚀 22841830 26302657 0.87
tpcds_q13/datafusion:vortex-compact 126706637 122361885 1.04
tpcds_q14/datafusion:vortex-compact 183868128 182505426 1.01
tpcds_q15/datafusion:vortex-compact 28894164 28088691 1.03
tpcds_q16/datafusion:vortex-compact 28339379 27758835 1.02
tpcds_q17/datafusion:vortex-compact 73190412 73921806 0.99
tpcds_q18/datafusion:vortex-compact 76394041 74585762 1.02
tpcds_q19/datafusion:vortex-compact 29955839 29317990 1.02
tpcds_q20/datafusion:vortex-compact 25848984 26501364 0.98
tpcds_q21/datafusion:vortex-compact 40988545 39972870 1.03
tpcds_q22/datafusion:vortex-compact 123287974 127250329 0.97
tpcds_q23/datafusion:vortex-compact 167264133 163466283 1.02
tpcds_q24/datafusion:vortex-compact 98090814 95044312 1.03
tpcds_q25/datafusion:vortex-compact 77939404 78714583 0.99
tpcds_q26/datafusion:vortex-compact 45153440 44602546 1.01
tpcds_q27/datafusion:vortex-compact 115895684 121026448 0.96
tpcds_q28/datafusion:vortex-compact 101720673 98540430 1.03
tpcds_q29/datafusion:vortex-compact 72853217 71772616 1.02
tpcds_q30/datafusion:vortex-compact 28970932 28450908 1.02
tpcds_q31/datafusion:vortex-compact 105859663 101138066 1.05
tpcds_q32/datafusion:vortex-compact 24874768 24156145 1.03
tpcds_q33/datafusion:vortex-compact 38670404 37292632 1.04
tpcds_q34/datafusion:vortex-compact 33702798 34404135 0.98
tpcds_q35/datafusion:vortex-compact 52795843 52374397 1.01
tpcds_q36/datafusion:vortex-compact 75399380 75471988 1.00
tpcds_q37/datafusion:vortex-compact 30587721 29738792 1.03
tpcds_q38/datafusion:vortex-compact 52732775 51176041 1.03
tpcds_q39/datafusion:vortex-compact 118974418 118189053 1.01
tpcds_q40/datafusion:vortex-compact 34782129 34469230 1.01
tpcds_q41/datafusion:vortex-compact 18030478 17514323 1.03
tpcds_q42/datafusion:vortex-compact 19224550 17871029 1.08
tpcds_q43/datafusion:vortex-compact 25036105 24742664 1.01
tpcds_q44/datafusion:vortex-compact 40639602 42025120 0.97
tpcds_q45/datafusion:vortex-compact 29752615 31365090 0.95
tpcds_q46/datafusion:vortex-compact 45990788 50278586 0.91
tpcds_q47/datafusion:vortex-compact 154322227 156505305 0.99
tpcds_q48/datafusion:vortex-compact 82239466 80269996 1.02
tpcds_q49/datafusion:vortex-compact 68307773 68229716 1.00
tpcds_q50/datafusion:vortex-compact 47509621 45561437 1.04
tpcds_q51/datafusion:vortex-compact 96830388 95167090 1.02
tpcds_q52/datafusion:vortex-compact 18874585 18940292 1.00
tpcds_q53/datafusion:vortex-compact 30060394 28072062 1.07
tpcds_q54/datafusion:vortex-compact 42104444 41721446 1.01
tpcds_q55/datafusion:vortex-compact 18345875 17887570 1.03
tpcds_q56/datafusion:vortex-compact 36263523 35272286 1.03
tpcds_q57/datafusion:vortex-compact 118546224 118303156 1.00
tpcds_q58/datafusion:vortex-compact 66676998 65940181 1.01
tpcds_q59/datafusion:vortex-compact 71077616 70159402 1.01
tpcds_q60/datafusion:vortex-compact 36712265 35448477 1.04
tpcds_q61/datafusion:vortex-compact 54119881 54208109 1.00
tpcds_q62/datafusion:vortex-compact 24368155 23975716 1.02
tpcds_q63/datafusion:vortex-compact 28534689 28166227 1.01
tpcds_q64/datafusion:vortex-compact 420207940 416847796 1.01
tpcds_q65/datafusion:vortex-compact 67186109 63362460 1.06
tpcds_q66/datafusion:vortex-compact 75895577 75784391 1.00
tpcds_q67/datafusion:vortex-compact 162733823 166178722 0.98
tpcds_q68/datafusion:vortex-compact 45687285 45943904 0.99
tpcds_q69/datafusion:vortex-compact 47793514 45943028 1.04
tpcds_q70/datafusion:vortex-compact 109252021 108631403 1.01
tpcds_q71/datafusion:vortex-compact 30239069 30294694 1.00
tpcds_q72/datafusion:vortex-compact 2140792643 2102441292 1.02
tpcds_q73/datafusion:vortex-compact 31263024 30622693 1.02
tpcds_q74/datafusion:vortex-compact 95243135 94686653 1.01
tpcds_q75/datafusion:vortex-compact 130516378 128682670 1.01
tpcds_q76/datafusion:vortex-compact 33663918 31883017 1.06
tpcds_q77/datafusion:vortex-compact 50234357 48688296 1.03
tpcds_q78/datafusion:vortex-compact 131509941 128351747 1.02
tpcds_q79/datafusion:vortex-compact 38258196 38605510 0.99
tpcds_q80/datafusion:vortex-compact 98768023 96422195 1.02
tpcds_q81/datafusion:vortex-compact 28215959 30612178 0.92
tpcds_q82/datafusion:vortex-compact 32414883 31745541 1.02
tpcds_q83/datafusion:vortex-compact 36853540 35951945 1.03
tpcds_q84/datafusion:vortex-compact 🚨 15555844 14126502 1.10
tpcds_q85/datafusion:vortex-compact 154208853 166175239 0.93
tpcds_q86/datafusion:vortex-compact 22263409 21502558 1.04
tpcds_q87/datafusion:vortex-compact 52380671 50380384 1.04
tpcds_q88/datafusion:vortex-compact 75998234 78368818 0.97
tpcds_q89/datafusion:vortex-compact 32243914 31231014 1.03
tpcds_q90/datafusion:vortex-compact 14952592 15117491 0.99
tpcds_q91/datafusion:vortex-compact 33144620 31861708 1.04
tpcds_q92/datafusion:vortex-compact 23795016 23812735 1.00
tpcds_q93/datafusion:vortex-compact 36097975 36827985 0.98
tpcds_q94/datafusion:vortex-compact 🚀 24300467 27386065 0.89
tpcds_q95/datafusion:vortex-compact 64249857 62466047 1.03
tpcds_q96/datafusion:vortex-compact 16888904 17215864 0.98
tpcds_q97/datafusion:vortex-compact 36326139 34387676 1.06
tpcds_q98/datafusion:vortex-compact 29503644 29319685 1.01
tpcds_q99/datafusion:vortex-compact 36389511 34910098 1.04
datafusion / parquet (1.014x ➖, 1↑ 1↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpcds_q01/datafusion:parquet 23984088 25328621 0.95
tpcds_q02/datafusion:parquet 41756637 41399050 1.01
tpcds_q03/datafusion:parquet 13755476 14179859 0.97
tpcds_q04/datafusion:parquet 304268664 307742349 0.99
tpcds_q05/datafusion:parquet 46438283 45153575 1.03
tpcds_q06/datafusion:parquet 23147318 22164456 1.04
tpcds_q07/datafusion:parquet 79431342 75818197 1.05
tpcds_q08/datafusion:parquet 30787946 28971590 1.06
tpcds_q09/datafusion:parquet 41040903 38750109 1.06
tpcds_q10/datafusion:parquet 75657062 70037252 1.08
tpcds_q11/datafusion:parquet 152278980 142924584 1.07
tpcds_q12/datafusion:parquet 18806955 18245549 1.03
tpcds_q13/datafusion:parquet 78167062 73860906 1.06
tpcds_q14/datafusion:parquet 159793902 160074678 1.00
tpcds_q15/datafusion:parquet 20946226 21957828 0.95
tpcds_q16/datafusion:parquet 29458556 30292590 0.97
tpcds_q17/datafusion:parquet 60325661 63537429 0.95
tpcds_q18/datafusion:parquet 109879801 109680294 1.00
tpcds_q19/datafusion:parquet 23664501 23561061 1.00
tpcds_q20/datafusion:parquet 17740449 16659478 1.06
tpcds_q21/datafusion:parquet 18830549 18604350 1.01
tpcds_q22/datafusion:parquet 161315659 148863008 1.08
tpcds_q23/datafusion:parquet 154995740 154468661 1.00
tpcds_q24/datafusion:parquet 88739131 87272565 1.02
tpcds_q25/datafusion:parquet 64501332 64937712 0.99
tpcds_q26/datafusion:parquet 64796961 68350536 0.95
tpcds_q27/datafusion:parquet 145956666 137285462 1.06
tpcds_q28/datafusion:parquet 44688547 44236628 1.01
tpcds_q29/datafusion:parquet 64665141 63408540 1.02
tpcds_q30/datafusion:parquet 34916899 32495824 1.07
tpcds_q31/datafusion:parquet 69251390 64788336 1.07
tpcds_q32/datafusion:parquet 17057172 16832832 1.01
tpcds_q33/datafusion:parquet 27573926 29433637 0.94
tpcds_q34/datafusion:parquet 22510836 22764112 0.99
tpcds_q35/datafusion:parquet 🚨 76844759 68600394 1.12
tpcds_q36/datafusion:parquet 58080671 57685709 1.01
tpcds_q37/datafusion:parquet 20121262 19881788 1.01
tpcds_q38/datafusion:parquet 42545264 42465597 1.00
tpcds_q39/datafusion:parquet 76034908 73838876 1.03
tpcds_q40/datafusion:parquet 24086454 24209971 0.99
tpcds_q41/datafusion:parquet 14334455 14738152 0.97
tpcds_q42/datafusion:parquet 11744891 11834442 0.99
tpcds_q43/datafusion:parquet 17106202 17247650 0.99
tpcds_q44/datafusion:parquet 33531905 33521254 1.00
tpcds_q45/datafusion:parquet 27839261 27333187 1.02
tpcds_q46/datafusion:parquet 32838157 31315437 1.05
tpcds_q47/datafusion:parquet 129418280 126949589 1.02
tpcds_q48/datafusion:parquet 70188185 68182201 1.03
tpcds_q49/datafusion:parquet 56822468 54775112 1.04
tpcds_q50/datafusion:parquet 43015621 42068153 1.02
tpcds_q51/datafusion:parquet 89032392 86601095 1.03
tpcds_q52/datafusion:parquet 13099565 12274535 1.07
tpcds_q53/datafusion:parquet 18411163 18303779 1.01
tpcds_q54/datafusion:parquet 33466828 32911209 1.02
tpcds_q55/datafusion:parquet 12250469 11481969 1.07
tpcds_q56/datafusion:parquet 28326516 29424289 0.96
tpcds_q57/datafusion:parquet 102795868 94113660 1.09
tpcds_q58/datafusion:parquet 56308159 54056981 1.04
tpcds_q59/datafusion:parquet 64005109 61856468 1.03
tpcds_q60/datafusion:parquet 28381046 28401670 1.00
tpcds_q61/datafusion:parquet 44127163 42907492 1.03
tpcds_q62/datafusion:parquet 25271979 25508636 0.99
tpcds_q63/datafusion:parquet 18393500 18254497 1.01
tpcds_q64/datafusion:parquet 305264751 316887229 0.96
tpcds_q65/datafusion:parquet 38813012 37735073 1.03
tpcds_q66/datafusion:parquet 71056965 69404031 1.02
tpcds_q67/datafusion:parquet 151351132 149337910 1.01
tpcds_q68/datafusion:parquet 31427409 31526422 1.00
tpcds_q69/datafusion:parquet 70009404 68520971 1.02
tpcds_q70/datafusion:parquet 33271410 35344218 0.94
tpcds_q71/datafusion:parquet 22342998 22883073 0.98
tpcds_q72/datafusion:parquet 576480460 584742883 0.99
tpcds_q73/datafusion:parquet 21155153 20981744 1.01
tpcds_q74/datafusion:parquet 83616664 84417166 0.99
tpcds_q75/datafusion:parquet 107780443 106392560 1.01
tpcds_q76/datafusion:parquet 30953087 29699221 1.04
tpcds_q77/datafusion:parquet 38752132 39555506 0.98
tpcds_q78/datafusion:parquet 107930933 111178781 0.97
tpcds_q79/datafusion:parquet 26920708 27251950 0.99
tpcds_q80/datafusion:parquet 77499247 78840136 0.98
tpcds_q81/datafusion:parquet 30026303 30034128 1.00
tpcds_q82/datafusion:parquet 19926330 18848377 1.06
tpcds_q83/datafusion:parquet 41097974 40763538 1.01
tpcds_q84/datafusion:parquet 40263639 40939325 0.98
tpcds_q85/datafusion:parquet 149695072 143906721 1.04
tpcds_q86/datafusion:parquet 🚀 14203139 16086067 0.88
tpcds_q87/datafusion:parquet 45905736 44346737 1.04
tpcds_q88/datafusion:parquet 61104839 59854269 1.02
tpcds_q89/datafusion:parquet 22166025 22217470 1.00
tpcds_q90/datafusion:parquet 14588534 13986221 1.04
tpcds_q91/datafusion:parquet 59894084 59075025 1.01
tpcds_q92/datafusion:parquet 17813011 17462367 1.02
tpcds_q93/datafusion:parquet 30670110 30790824 1.00
tpcds_q94/datafusion:parquet 22295367 20308315 1.10
tpcds_q95/datafusion:parquet 60963922 61371895 0.99
tpcds_q96/datafusion:parquet 12698810 12116336 1.05
tpcds_q97/datafusion:parquet 31476729 30833100 1.02
tpcds_q98/datafusion:parquet 21510471 20890847 1.03
tpcds_q99/datafusion:parquet 26798052 25596379 1.05
duckdb / vortex-file-compressed (1.013x ➖, 0↑ 4↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpcds_q01/duckdb:vortex-file-compressed 23295518 24431288 0.95
tpcds_q02/duckdb:vortex-file-compressed 34521757 36750869 0.94
tpcds_q03/duckdb:vortex-file-compressed 15839218 15518557 1.02
tpcds_q04/duckdb:vortex-file-compressed 93633268 95602690 0.98
tpcds_q05/duckdb:vortex-file-compressed 🚨 35086610 31267921 1.12
tpcds_q06/duckdb:vortex-file-compressed 33169399 33064343 1.00
tpcds_q07/duckdb:vortex-file-compressed 21175234 21600949 0.98
tpcds_q08/duckdb:vortex-file-compressed 26783197 26375873 1.02
tpcds_q09/duckdb:vortex-file-compressed 17231969 16846063 1.02
tpcds_q10/duckdb:vortex-file-compressed 37031066 36789279 1.01
tpcds_q11/duckdb:vortex-file-compressed 65599663 64219970 1.02
tpcds_q12/duckdb:vortex-file-compressed 16059181 15971358 1.01
tpcds_q13/duckdb:vortex-file-compressed 33298427 34403213 0.97
tpcds_q14/duckdb:vortex-file-compressed 96806590 95518306 1.01
tpcds_q15/duckdb:vortex-file-compressed 26653208 26906450 0.99
tpcds_q16/duckdb:vortex-file-compressed 25866702 25724462 1.01
tpcds_q17/duckdb:vortex-file-compressed 44182457 44676933 0.99
tpcds_q18/duckdb:vortex-file-compressed 37326703 37534793 0.99
tpcds_q19/duckdb:vortex-file-compressed 32888329 32122115 1.02
tpcds_q20/duckdb:vortex-file-compressed 17535696 16898486 1.04
tpcds_q21/duckdb:vortex-file-compressed 17396612 16717850 1.04
tpcds_q22/duckdb:vortex-file-compressed 78313544 73927748 1.06
tpcds_q23/duckdb:vortex-file-compressed 97775779 97959268 1.00
tpcds_q24/duckdb:vortex-file-compressed 48672503 47001205 1.04
tpcds_q25/duckdb:vortex-file-compressed 41069302 41899437 0.98
tpcds_q26/duckdb:vortex-file-compressed 19089857 20846230 0.92
tpcds_q27/duckdb:vortex-file-compressed 24393681 23869788 1.02
tpcds_q28/duckdb:vortex-file-compressed 11952968 11606323 1.03
tpcds_q29/duckdb:vortex-file-compressed 40090570 39598364 1.01
tpcds_q30/duckdb:vortex-file-compressed 25176555 25074510 1.00
tpcds_q31/duckdb:vortex-file-compressed 31120699 29557407 1.05
tpcds_q32/duckdb:vortex-file-compressed 13334902 12893341 1.03
tpcds_q33/duckdb:vortex-file-compressed 23961657 24535728 0.98
tpcds_q34/duckdb:vortex-file-compressed 23934028 24583310 0.97
tpcds_q35/duckdb:vortex-file-compressed 65282210 65173625 1.00
tpcds_q36/duckdb:vortex-file-compressed 25026448 25464718 0.98
tpcds_q37/duckdb:vortex-file-compressed 19533278 19871574 0.98
tpcds_q38/duckdb:vortex-file-compressed 37824162 37643059 1.00
tpcds_q39/duckdb:vortex-file-compressed 35563141 36132833 0.98
tpcds_q40/duckdb:vortex-file-compressed 20215543 19983958 1.01
tpcds_q41/duckdb:vortex-file-compressed 9181241 9015385 1.02
tpcds_q42/duckdb:vortex-file-compressed 14058304 13288999 1.06
tpcds_q43/duckdb:vortex-file-compressed 🚨 21182333 17022380 1.24
tpcds_q44/duckdb:vortex-file-compressed 21110059 20332313 1.04
tpcds_q45/duckdb:vortex-file-compressed 30067795 29631676 1.01
tpcds_q46/duckdb:vortex-file-compressed 32608400 32841455 0.99
tpcds_q47/duckdb:vortex-file-compressed 51780069 51766329 1.00
tpcds_q48/duckdb:vortex-file-compressed 29480641 28555437 1.03
tpcds_q49/duckdb:vortex-file-compressed 35951149 34791610 1.03
tpcds_q50/duckdb:vortex-file-compressed 25777349 27613917 0.93
tpcds_q51/duckdb:vortex-file-compressed 105111418 106093788 0.99
tpcds_q52/duckdb:vortex-file-compressed 14033925 14406521 0.97
tpcds_q53/duckdb:vortex-file-compressed 22528918 22210687 1.01
tpcds_q54/duckdb:vortex-file-compressed 28704150 28929353 0.99
tpcds_q55/duckdb:vortex-file-compressed 12382275 12490507 0.99
tpcds_q56/duckdb:vortex-file-compressed 25955809 25685528 1.01
tpcds_q57/duckdb:vortex-file-compressed 43332123 42420561 1.02
tpcds_q58/duckdb:vortex-file-compressed 29979730 30360862 0.99
tpcds_q59/duckdb:vortex-file-compressed 57212518 56590239 1.01
tpcds_q60/duckdb:vortex-file-compressed 26797261 26130706 1.03
tpcds_q61/duckdb:vortex-file-compressed 32083140 31859786 1.01
tpcds_q62/duckdb:vortex-file-compressed 16385162 15697232 1.04
tpcds_q63/duckdb:vortex-file-compressed 20988419 20580495 1.02
tpcds_q64/duckdb:vortex-file-compressed 94828100 95740153 0.99
tpcds_q65/duckdb:vortex-file-compressed 22372591 22708584 0.99
tpcds_q66/duckdb:vortex-file-compressed 31265283 30151326 1.04
tpcds_q67/duckdb:vortex-file-compressed 141067052 139645131 1.01
tpcds_q68/duckdb:vortex-file-compressed 29879494 31270052 0.96
tpcds_q69/duckdb:vortex-file-compressed 39801097 39855894 1.00
tpcds_q70/duckdb:vortex-file-compressed 39398001 37908047 1.04
tpcds_q71/duckdb:vortex-file-compressed 21204442 20874135 1.02
tpcds_q72/duckdb:vortex-file-compressed 170821492 168468990 1.01
tpcds_q73/duckdb:vortex-file-compressed 26148580 25054419 1.04
tpcds_q74/duckdb:vortex-file-compressed 42718108 41655912 1.03
tpcds_q75/duckdb:vortex-file-compressed 49677446 49894008 1.00
tpcds_q76/duckdb:vortex-file-compressed 21728740 20717842 1.05
tpcds_q77/duckdb:vortex-file-compressed 25071965 24469340 1.02
tpcds_q78/duckdb:vortex-file-compressed 82501371 78319505 1.05
tpcds_q79/duckdb:vortex-file-compressed 24349021 24859538 0.98
tpcds_q80/duckdb:vortex-file-compressed 49992950 47789286 1.05
tpcds_q81/duckdb:vortex-file-compressed 29195822 29808657 0.98
tpcds_q82/duckdb:vortex-file-compressed 🚨 51053191 45646936 1.12
tpcds_q83/duckdb:vortex-file-compressed 25641718 25648194 1.00
tpcds_q84/duckdb:vortex-file-compressed 17548739 17534231 1.00
tpcds_q85/duckdb:vortex-file-compressed 39981909 40210725 0.99
tpcds_q86/duckdb:vortex-file-compressed 16620775 16137973 1.03
tpcds_q87/duckdb:vortex-file-compressed 40752336 41925725 0.97
tpcds_q88/duckdb:vortex-file-compressed 53514235 53301786 1.00
tpcds_q89/duckdb:vortex-file-compressed 22273192 21968012 1.01
tpcds_q90/duckdb:vortex-file-compressed 10918333 10115336 1.08
tpcds_q91/duckdb:vortex-file-compressed 🚨 24192158 21595625 1.12
tpcds_q92/duckdb:vortex-file-compressed 16998379 17554397 0.97
tpcds_q93/duckdb:vortex-file-compressed 30043549 30207989 0.99
tpcds_q94/duckdb:vortex-file-compressed 21203686 21342546 0.99
tpcds_q95/duckdb:vortex-file-compressed 162887289 157123032 1.04
tpcds_q96/duckdb:vortex-file-compressed 12951431 12744328 1.02
tpcds_q97/duckdb:vortex-file-compressed 41144168 38399668 1.07
tpcds_q98/duckdb:vortex-file-compressed 20294024 19075835 1.06
tpcds_q99/duckdb:vortex-file-compressed 22819188 23368481 0.98
duckdb / vortex-compact (1.017x ➖, 1↑ 2↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpcds_q01/duckdb:vortex-compact 25791666 25611747 1.01
tpcds_q02/duckdb:vortex-compact 39252577 39457198 0.99
tpcds_q03/duckdb:vortex-compact 47342190 46847862 1.01
tpcds_q04/duckdb:vortex-compact 102969351 104352500 0.99
tpcds_q05/duckdb:vortex-compact 42522725 42826686 0.99
tpcds_q06/duckdb:vortex-compact 39282593 38799779 1.01
tpcds_q07/duckdb:vortex-compact 36286068 35445862 1.02
tpcds_q08/duckdb:vortex-compact 40841284 40718008 1.00
tpcds_q09/duckdb:vortex-compact 21524394 19829529 1.09
tpcds_q10/duckdb:vortex-compact 53098335 52652697 1.01
tpcds_q11/duckdb:vortex-compact 72919976 70976918 1.03
tpcds_q12/duckdb:vortex-compact 22378138 21974929 1.02
tpcds_q13/duckdb:vortex-compact 53453300 52777699 1.01
tpcds_q14/duckdb:vortex-compact 120768387 115218908 1.05
tpcds_q15/duckdb:vortex-compact 29876990 31433415 0.95
tpcds_q16/duckdb:vortex-compact 27416285 27955673 0.98
tpcds_q17/duckdb:vortex-compact 55734139 55656460 1.00
tpcds_q18/duckdb:vortex-compact 50219979 48861803 1.03
tpcds_q19/duckdb:vortex-compact 44008249 45310084 0.97
tpcds_q20/duckdb:vortex-compact 21149622 21002089 1.01
tpcds_q21/duckdb:vortex-compact 🚨 21382995 17673758 1.21
tpcds_q22/duckdb:vortex-compact 83611290 76404720 1.09
tpcds_q23/duckdb:vortex-compact 109022612 108761815 1.00
tpcds_q24/duckdb:vortex-compact 58515110 58324962 1.00
tpcds_q25/duckdb:vortex-compact 55062675 53758618 1.02
tpcds_q26/duckdb:vortex-compact 31169470 31638491 0.99
tpcds_q27/duckdb:vortex-compact 36434483 38812174 0.94
tpcds_q28/duckdb:vortex-compact 34142004 35369716 0.97
tpcds_q29/duckdb:vortex-compact 56041627 52714645 1.06
tpcds_q30/duckdb:vortex-compact 29649998 31709218 0.94
tpcds_q31/duckdb:vortex-compact 36373475 37302630 0.98
tpcds_q32/duckdb:vortex-compact 21025198 20537202 1.02
tpcds_q33/duckdb:vortex-compact 33742662 33322360 1.01
tpcds_q34/duckdb:vortex-compact 31286718 30410812 1.03
tpcds_q35/duckdb:vortex-compact 75599209 75327645 1.00
tpcds_q36/duckdb:vortex-compact 37870124 36535706 1.04
tpcds_q37/duckdb:vortex-compact 23603728 22987225 1.03
tpcds_q38/duckdb:vortex-compact 43570619 41335897 1.05
tpcds_q39/duckdb:vortex-compact 37966070 38008307 1.00
tpcds_q40/duckdb:vortex-compact 25933757 23656596 1.10
tpcds_q41/duckdb:vortex-compact 10990216 10753363 1.02
tpcds_q42/duckdb:vortex-compact 19919146 18925727 1.05
tpcds_q43/duckdb:vortex-compact 31728977 30025022 1.06
tpcds_q44/duckdb:vortex-compact 30849280 29650073 1.04
tpcds_q45/duckdb:vortex-compact 🚨 39422805 35138776 1.12
tpcds_q46/duckdb:vortex-compact 47000708 44441545 1.06
tpcds_q47/duckdb:vortex-compact 63246857 61813668 1.02
tpcds_q48/duckdb:vortex-compact 47551186 45988800 1.03
tpcds_q49/duckdb:vortex-compact 🚀 46208999 52141487 0.89
tpcds_q50/duckdb:vortex-compact 38611435 37911839 1.02
tpcds_q51/duckdb:vortex-compact 123009654 114106013 1.08
tpcds_q52/duckdb:vortex-compact 19125240 19625969 0.97
tpcds_q53/duckdb:vortex-compact 33391008 32148963 1.04
tpcds_q54/duckdb:vortex-compact 39667677 36721411 1.08
tpcds_q55/duckdb:vortex-compact 19102857 19009113 1.00
tpcds_q56/duckdb:vortex-compact 33512657 33048118 1.01
tpcds_q57/duckdb:vortex-compact 45742862 44942581 1.02
tpcds_q58/duckdb:vortex-compact 40363768 38393409 1.05
tpcds_q59/duckdb:vortex-compact 66596685 67582982 0.99
tpcds_q60/duckdb:vortex-compact 35031115 35159890 1.00
tpcds_q61/duckdb:vortex-compact 54740949 55201130 0.99
tpcds_q62/duckdb:vortex-compact 22098054 21473988 1.03
tpcds_q63/duckdb:vortex-compact 31487291 33044642 0.95
tpcds_q64/duckdb:vortex-compact 132436873 123154161 1.08
tpcds_q65/duckdb:vortex-compact 30505368 30904702 0.99
tpcds_q66/duckdb:vortex-compact 39473133 35952710 1.10
tpcds_q67/duckdb:vortex-compact 158168455 150998101 1.05
tpcds_q68/duckdb:vortex-compact 47556523 45922206 1.04
tpcds_q69/duckdb:vortex-compact 55823793 55107393 1.01
tpcds_q70/duckdb:vortex-compact 58804431 56311481 1.04
tpcds_q71/duckdb:vortex-compact 30196494 30269331 1.00
tpcds_q72/duckdb:vortex-compact 187769182 187351868 1.00
tpcds_q73/duckdb:vortex-compact 28580588 28739655 0.99
tpcds_q74/duckdb:vortex-compact 48489472 49589684 0.98
tpcds_q75/duckdb:vortex-compact 64140195 59906986 1.07
tpcds_q76/duckdb:vortex-compact 28288808 30256625 0.93
tpcds_q77/duckdb:vortex-compact 38956733 38942903 1.00
tpcds_q78/duckdb:vortex-compact 87729271 86894991 1.01
tpcds_q79/duckdb:vortex-compact 40506221 43713334 0.93
tpcds_q80/duckdb:vortex-compact 74394844 70463392 1.06
tpcds_q81/duckdb:vortex-compact 34302670 33841044 1.01
tpcds_q82/duckdb:vortex-compact 50913272 50495813 1.01
tpcds_q83/duckdb:vortex-compact 36186537 35970809 1.01
tpcds_q84/duckdb:vortex-compact 21968331 20668621 1.06
tpcds_q85/duckdb:vortex-compact 52209914 56157420 0.93
tpcds_q86/duckdb:vortex-compact 21431213 20778229 1.03
tpcds_q87/duckdb:vortex-compact 46085313 44926702 1.03
tpcds_q88/duckdb:vortex-compact 77571887 77000884 1.01
tpcds_q89/duckdb:vortex-compact 32120696 32908270 0.98
tpcds_q90/duckdb:vortex-compact 11251819 11003022 1.02
tpcds_q91/duckdb:vortex-compact 41204020 40677415 1.01
tpcds_q92/duckdb:vortex-compact 44899467 46306893 0.97
tpcds_q93/duckdb:vortex-compact 33739278 32255482 1.05
tpcds_q94/duckdb:vortex-compact 28696517 26545595 1.08
tpcds_q95/duckdb:vortex-compact 166316272 163813458 1.02
tpcds_q96/duckdb:vortex-compact 16408623 16747255 0.98
tpcds_q97/duckdb:vortex-compact 43165360 41082010 1.05
tpcds_q98/duckdb:vortex-compact 28380501 28496728 1.00
tpcds_q99/duckdb:vortex-compact 27969879 25997309 1.08
duckdb / parquet (1.009x ➖, 0↑ 1↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpcds_q01/duckdb:parquet 30151928 28655634 1.05
tpcds_q02/duckdb:parquet 24786808 24457664 1.01
tpcds_q03/duckdb:parquet 13136844 12568976 1.05
tpcds_q04/duckdb:parquet 170271623 169053980 1.01
tpcds_q05/duckdb:parquet 31849569 32644922 0.98
tpcds_q06/duckdb:parquet 33567378 34042956 0.99
tpcds_q07/duckdb:parquet 24055843 24386945 0.99
tpcds_q08/duckdb:parquet 29710348 29749460 1.00
tpcds_q09/duckdb:parquet 45028731 44544992 1.01
tpcds_q10/duckdb:parquet 37562431 38097980 0.99
tpcds_q11/duckdb:parquet 89764492 86636400 1.04
tpcds_q12/duckdb:parquet 17120302 17006498 1.01
tpcds_q13/duckdb:parquet 35306902 34621164 1.02
tpcds_q14/duckdb:parquet 101214651 100912580 1.00
tpcds_q15/duckdb:parquet 30881462 30259093 1.02
tpcds_q16/duckdb:parquet 22813878 22682042 1.01
tpcds_q17/duckdb:parquet 38934960 39994055 0.97
tpcds_q18/duckdb:parquet 48214605 48685354 0.99
tpcds_q19/duckdb:parquet 32132418 31521925 1.02
tpcds_q20/duckdb:parquet 18719630 18553483 1.01
tpcds_q21/duckdb:parquet 11768467 11754727 1.00
tpcds_q22/duckdb:parquet 73367351 72179523 1.02
tpcds_q23/duckdb:parquet 81961387 83947895 0.98
tpcds_q24/duckdb:parquet 47198611 46869374 1.01
tpcds_q25/duckdb:parquet 35919693 35662864 1.01
tpcds_q26/duckdb:parquet 40211052 39729917 1.01
tpcds_q27/duckdb:parquet 53681265 52817500 1.02
tpcds_q28/duckdb:parquet 42918973 42952302 1.00
tpcds_q29/duckdb:parquet 38487639 36521806 1.05
tpcds_q30/duckdb:parquet 37984669 37296412 1.02
tpcds_q31/duckdb:parquet 28555779 28228695 1.01
tpcds_q32/duckdb:parquet 12368733 12494654 0.99
tpcds_q33/duckdb:parquet 23257823 23889825 0.97
tpcds_q34/duckdb:parquet 23076678 24107109 0.96
tpcds_q35/duckdb:parquet 62307605 61636867 1.01
tpcds_q36/duckdb:parquet 23090608 22646943 1.02
tpcds_q37/duckdb:parquet 14814004 15146154 0.98
tpcds_q38/duckdb:parquet 40414040 36869889 1.10
tpcds_q39/duckdb:parquet 32894645 32780602 1.00
tpcds_q40/duckdb:parquet 19932310 19828202 1.01
tpcds_q41/duckdb:parquet 8978878 8800079 1.02
tpcds_q42/duckdb:parquet 12910189 12751935 1.01
tpcds_q43/duckdb:parquet 17667401 17871384 0.99
tpcds_q44/duckdb:parquet 26718308 26073996 1.02
tpcds_q45/duckdb:parquet 28444043 30235738 0.94
tpcds_q46/duckdb:parquet 47400865 47577869 1.00
tpcds_q47/duckdb:parquet 50028157 49578154 1.01
tpcds_q48/duckdb:parquet 32381877 32277812 1.00
tpcds_q49/duckdb:parquet 29147336 28208484 1.03
tpcds_q50/duckdb:parquet 26257106 26141013 1.00
tpcds_q51/duckdb:parquet 104968767 104605447 1.00
tpcds_q52/duckdb:parquet 13007984 12972862 1.00
tpcds_q53/duckdb:parquet 19238928 18874002 1.02
tpcds_q54/duckdb:parquet 29282258 28982881 1.01
tpcds_q55/duckdb:parquet 12610429 12641918 1.00
tpcds_q56/duckdb:parquet 24356938 23783610 1.02
tpcds_q57/duckdb:parquet 38072401 38102313 1.00
tpcds_q58/duckdb:parquet 25931279 26326836 0.98
tpcds_q59/duckdb:parquet 36408460 36768978 0.99
tpcds_q60/duckdb:parquet 24380916 24709722 0.99
tpcds_q61/duckdb:parquet 34171539 33705124 1.01
tpcds_q62/duckdb:parquet 12706885 12880531 0.99
tpcds_q63/duckdb:parquet 18260286 17916147 1.02
tpcds_q64/duckdb:parquet 81458644 81183119 1.00
tpcds_q65/duckdb:parquet 22927746 22277536 1.03
tpcds_q66/duckdb:parquet 31141862 29240384 1.07
tpcds_q67/duckdb:parquet 137310513 134990944 1.02
tpcds_q68/duckdb:parquet 39604380 39702858 1.00
tpcds_q69/duckdb:parquet 39277236 38442271 1.02
tpcds_q70/duckdb:parquet 22244923 23436464 0.95
tpcds_q71/duckdb:parquet 23055410 23369916 0.99
tpcds_q72/duckdb:parquet 169706969 167564485 1.01
tpcds_q73/duckdb:parquet 21077026 20085104 1.05
tpcds_q74/duckdb:parquet 125564085 127415129 0.99
tpcds_q75/duckdb:parquet 57099205 57152261 1.00
tpcds_q76/duckdb:parquet 22188061 21744725 1.02
tpcds_q77/duckdb:parquet 25149047 25702085 0.98
tpcds_q78/duckdb:parquet 77532948 77204904 1.00
tpcds_q79/duckdb:parquet 30425684 30546676 1.00
tpcds_q80/duckdb:parquet 43110877 43372256 0.99
tpcds_q81/duckdb:parquet 35792580 35757454 1.00
tpcds_q82/duckdb:parquet 16772578 16347302 1.03
tpcds_q83/duckdb:parquet 18907959 18081641 1.05
tpcds_q84/duckdb:parquet 20441959 20916175 0.98
tpcds_q85/duckdb:parquet 42067649 42082417 1.00
tpcds_q86/duckdb:parquet 13654139 13825977 0.99
tpcds_q87/duckdb:parquet 39255869 39857852 0.98
tpcds_q88/duckdb:parquet 53341965 53374864 1.00
tpcds_q89/duckdb:parquet 22017186 22021858 1.00
tpcds_q90/duckdb:parquet 8473910 8167693 1.04
tpcds_q91/duckdb:parquet 24702568 25478552 0.97
tpcds_q92/duckdb:parquet 13538754 13201991 1.03
tpcds_q93/duckdb:parquet 32378912 32079296 1.01
tpcds_q94/duckdb:parquet 18986265 17940656 1.06
tpcds_q95/duckdb:parquet 149884580 151250891 0.99
tpcds_q96/duckdb:parquet 10656261 10577193 1.01
tpcds_q97/duckdb:parquet 38804540 38271801 1.01
tpcds_q98/duckdb:parquet 23428569 24401547 0.96
tpcds_q99/duckdb:parquet 🚨 27132856 19984052 1.36
duckdb / duckdb (1.014x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpcds_q01/duckdb:duckdb 21995376 22257715 0.99
tpcds_q02/duckdb:duckdb 19258359 19407928 0.99
tpcds_q03/duckdb:duckdb 10559764 10625673 0.99
tpcds_q04/duckdb:duckdb 179283433 179146653 1.00
tpcds_q05/duckdb:duckdb 21834066 20918209 1.04
tpcds_q06/duckdb:duckdb 28592447 28524835 1.00
tpcds_q07/duckdb:duckdb 40393017 39815507 1.01
tpcds_q08/duckdb:duckdb 120568754 121586086 0.99
tpcds_q09/duckdb:duckdb 20570825 20571113 1.00
tpcds_q10/duckdb:duckdb 27668432 27776361 1.00
tpcds_q11/duckdb:duckdb 95229604 96197880 0.99
tpcds_q12/duckdb:duckdb 14115101 14192037 0.99
tpcds_q13/duckdb:duckdb 26633060 26048534 1.02
tpcds_q14/duckdb:duckdb 94234259 96198009 0.98
tpcds_q15/duckdb:duckdb 25525409 25083524 1.02
tpcds_q16/duckdb:duckdb 20201259 19108668 1.06
tpcds_q17/duckdb:duckdb 25525804 23737824 1.08
tpcds_q18/duckdb:duckdb 47894671 47246202 1.01
tpcds_q19/duckdb:duckdb 19686021 18960553 1.04
tpcds_q20/duckdb:duckdb 14366223 14317719 1.00
tpcds_q21/duckdb:duckdb 8175816 7561916 1.08
tpcds_q22/duckdb:duckdb 63096471 66454658 0.95
tpcds_q23/duckdb:duckdb 79621598 76531881 1.04
tpcds_q24/duckdb:duckdb 26055992 25632722 1.02
tpcds_q25/duckdb:duckdb 18836822 18641610 1.01
tpcds_q26/duckdb:duckdb 28227425 28323496 1.00
tpcds_q27/duckdb:duckdb 41618307 41522147 1.00
tpcds_q28/duckdb:duckdb 26397532 26070610 1.01
tpcds_q29/duckdb:duckdb 23102598 22950397 1.01
tpcds_q30/duckdb:duckdb 31256819 30731302 1.02
tpcds_q31/duckdb:duckdb 51797641 51687792 1.00
tpcds_q32/duckdb:duckdb 8122460 8142554 1.00
tpcds_q33/duckdb:duckdb 16975443 16306377 1.04
tpcds_q34/duckdb:duckdb 18959846 17288274 1.10
tpcds_q35/duckdb:duckdb 37849351 35564866 1.06
tpcds_q36/duckdb:duckdb 71175519 71241415 1.00
tpcds_q37/duckdb:duckdb 8841375 8583268 1.03
tpcds_q38/duckdb:duckdb 34051431 33054763 1.03
tpcds_q39/duckdb:duckdb 28555265 28301767 1.01
tpcds_q40/duckdb:duckdb 15178762 14720176 1.03
tpcds_q41/duckdb:duckdb 9434558 9343105 1.01
tpcds_q42/duckdb:duckdb 9416834 8808656 1.07
tpcds_q43/duckdb:duckdb 15135336 14325688 1.06
tpcds_q44/duckdb:duckdb 15314567 15676172 0.98
tpcds_q45/duckdb:duckdb 18978497 18638369 1.02
tpcds_q46/duckdb:duckdb 38354369 38334264 1.00
tpcds_q47/duckdb:duckdb 46366856 47138148 0.98
tpcds_q48/duckdb:duckdb 25663265 24528234 1.05
tpcds_q49/duckdb:duckdb 21047576 20184018 1.04
tpcds_q50/duckdb:duckdb 16046636 15820017 1.01
tpcds_q51/duckdb:duckdb 102359792 99482260 1.03
tpcds_q52/duckdb:duckdb 8928260 8947113 1.00
tpcds_q53/duckdb:duckdb 17882096 17465884 1.02
tpcds_q54/duckdb:duckdb 19369200 18728670 1.03
tpcds_q55/duckdb:duckdb 8795694 8843992 0.99
tpcds_q56/duckdb:duckdb 16953533 16864086 1.01
tpcds_q57/duckdb:duckdb 36299912 36314773 1.00
tpcds_q58/duckdb:duckdb 17605104 16700396 1.05
tpcds_q59/duckdb:duckdb 37081945 36483698 1.02
tpcds_q60/duckdb:duckdb 18342735 18748127 0.98
tpcds_q61/duckdb:duckdb 16951958 17027001 1.00
tpcds_q62/duckdb:duckdb 10122009 9645869 1.05
tpcds_q63/duckdb:duckdb 16912383 16836904 1.00
tpcds_q64/duckdb:duckdb 59052947 58671560 1.01
tpcds_q65/duckdb:duckdb 36748587 36829250 1.00
tpcds_q66/duckdb:duckdb 27349536 25833436 1.06
tpcds_q67/duckdb:duckdb 134507701 131808372 1.02
tpcds_q68/duckdb:duckdb 26855977 26836791 1.00
tpcds_q69/duckdb:duckdb 28305877 29099479 0.97
tpcds_q70/duckdb:duckdb 16664029 17256177 0.97
tpcds_q71/duckdb:duckdb 17187638 16434897 1.05
tpcds_q72/duckdb:duckdb 46726516 46096562 1.01
tpcds_q73/duckdb:duckdb 14299245 13469629 1.06
tpcds_q74/duckdb:duckdb 147105133 146053163 1.01
tpcds_q75/duckdb:duckdb 46564856 45789735 1.02
tpcds_q76/duckdb:duckdb 14523110 14189084 1.02
tpcds_q77/duckdb:duckdb 16115146 15708768 1.03
tpcds_q78/duckdb:duckdb 66528192 65032068 1.02
tpcds_q79/duckdb:duckdb 20181608 19818314 1.02
tpcds_q80/duckdb:duckdb 31693939 31372382 1.01
tpcds_q81/duckdb:duckdb 41101893 40551229 1.01
tpcds_q82/duckdb:duckdb 10463918 9953774 1.05
tpcds_q83/duckdb:duckdb 10921446 10872452 1.00
tpcds_q84/duckdb:duckdb 14840032 14459932 1.03
tpcds_q85/duckdb:duckdb 25358708 26317326 0.96
tpcds_q86/duckdb:duckdb 12263022 11962279 1.03
tpcds_q87/duckdb:duckdb 36032875 35327807 1.02
tpcds_q88/duckdb:duckdb 30555149 30425375 1.00
tpcds_q89/duckdb:duckdb 22060870 22308662 0.99
tpcds_q90/duckdb:duckdb 6237676 6094159 1.02
tpcds_q91/duckdb:duckdb 15306805 15635492 0.98
tpcds_q92/duckdb:duckdb 9937543 10163573 0.98
tpcds_q93/duckdb:duckdb 23574802 23699420 0.99
tpcds_q94/duckdb:duckdb 15098780 14826840 1.02
tpcds_q95/duckdb:duckdb 112942015 121720738 0.93
tpcds_q96/duckdb:duckdb 5863902 5723811 1.02
tpcds_q97/duckdb:duckdb 34408266 32026362 1.07
tpcds_q98/duckdb:duckdb 21234856 21356614 0.99
tpcds_q99/duckdb:duckdb 16866043 16622000 1.01

File Size Changes (6 files changed, -0.0% overall, 2↑ 4↓)
File Scale Format Base HEAD Change %
customer_address.vortex 1.0 vortex-file-compressed 826.07 KB 826.65 KB +592 B +0.1%
item.vortex 1.0 vortex-file-compressed 1.66 MB 1.66 MB +336 B +0.0%
item.vortex 1.0 vortex-compact 994.64 KB 994.25 KB 408 B -0.0%
catalog_page.vortex 1.0 vortex-file-compressed 572.50 KB 572.19 KB 320 B -0.1%
time_dim.vortex 1.0 vortex-file-compressed 381.80 KB 380.52 KB 1.28 KB -0.3%
customer.vortex 1.0 vortex-file-compressed 4.27 MB 4.18 MB 91.77 KB -2.1%

Totals:

  • vortex-compact: 207.50 MB → 207.50 MB (-0.0%)
  • vortex-file-compressed: 271.85 MB → 271.76 MB (-0.0%)

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Benchmarks: FineWeb S3

Verdict: No clear signal (low confidence)
Attributed Vortex impact: +14.7%
Engines: DataFusion No clear signal (+27.3%, environment too noisy confidence) · DuckDB No clear signal (+3.4%, environment too noisy confidence)
Vortex (geomean): 1.136x ➖
Parquet (geomean): 0.990x ➖
Shifts: Parquet (control) -1.0% · Median polish -2.0%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.002x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
fineweb_q00/datafusion:vortex-file-compressed 32592175 31362255 1.04
fineweb_q01/datafusion:vortex-file-compressed 779070494 671599512 1.16
fineweb_q02/datafusion:vortex-file-compressed 727868168 795731668 0.91
fineweb_q03/datafusion:vortex-file-compressed 1297447953 1109884652 1.17
fineweb_q04/datafusion:vortex-file-compressed 1167663767 1184710181 0.99
fineweb_q05/datafusion:vortex-file-compressed 1103674170 1102117018 1.00
fineweb_q06/datafusion:vortex-file-compressed 1302105114 1376910705 0.95
fineweb_q07/datafusion:vortex-file-compressed 1137000593 1384522311 0.82
fineweb_q08/datafusion:vortex-file-compressed 616811030 599861451 1.03
datafusion / vortex-compact (1.560x ❌, 0↑ 4↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
fineweb_q00/datafusion:vortex-compact 🚨 146391406 31875124 4.59
fineweb_q01/datafusion:vortex-compact 🚨 995174084 493717483 2.02
fineweb_q02/datafusion:vortex-compact 🚨 1564632968 581872268 2.69
fineweb_q03/datafusion:vortex-compact 1537956547 1332737335 1.15
fineweb_q04/datafusion:vortex-compact 1706812625 1636620965 1.04
fineweb_q05/datafusion:vortex-compact 1381759244 1348956714 1.02
fineweb_q06/datafusion:vortex-compact 1335106175 1268465069 1.05
fineweb_q07/datafusion:vortex-compact 1296135157 1154713578 1.12
fineweb_q08/datafusion:vortex-compact 🚨 617112212 409598527 1.51
datafusion / parquet (0.982x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
fineweb_q00/datafusion:parquet 1416152956 1379756222 1.03
fineweb_q01/datafusion:parquet 1848499789 2072622254 0.89
fineweb_q02/datafusion:parquet 2002963483 2224143179 0.90
fineweb_q03/datafusion:parquet 1917125755 2042489843 0.94
fineweb_q04/datafusion:parquet 2211167814 2073008858 1.07
fineweb_q05/datafusion:parquet 2163837636 2454711764 0.88
fineweb_q06/datafusion:parquet 2186985198 2185944318 1.00
fineweb_q07/datafusion:parquet 2152475423 2066167563 1.04
fineweb_q08/datafusion:parquet 2318127478 2070722314 1.12
duckdb / vortex-file-compressed (0.952x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
fineweb_q00/duckdb:vortex-file-compressed 96552750 92309053 1.05
fineweb_q01/duckdb:vortex-file-compressed 860454772 1031486046 0.83
fineweb_q02/duckdb:vortex-file-compressed 856654363 1051658648 0.81
fineweb_q03/duckdb:vortex-file-compressed 1542567897 1715812361 0.90
fineweb_q04/duckdb:vortex-file-compressed 1578762098 1416244560 1.11
fineweb_q05/duckdb:vortex-file-compressed 1520778737 1488384021 1.02
fineweb_q06/duckdb:vortex-file-compressed 1697060534 1829412986 0.93
fineweb_q07/duckdb:vortex-file-compressed 1427377993 1591953699 0.90
fineweb_q08/duckdb:vortex-file-compressed 640094530 603725599 1.06
duckdb / vortex-compact (1.120x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
fineweb_q00/duckdb:vortex-compact 98059344 111639291 0.88
fineweb_q01/duckdb:vortex-compact 873162155 702064957 1.24
fineweb_q02/duckdb:vortex-compact 898959939 692483234 1.30
fineweb_q03/duckdb:vortex-compact 1824077845 1657427267 1.10
fineweb_q04/duckdb:vortex-compact 2063477484 1809163568 1.14
fineweb_q05/duckdb:vortex-compact 1757436045 1694465380 1.04
fineweb_q06/duckdb:vortex-compact 1887955917 1628645693 1.16
fineweb_q07/duckdb:vortex-compact 1592401674 1472394598 1.08
fineweb_q08/duckdb:vortex-compact 572819441 478662539 1.20
duckdb / parquet (0.998x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
fineweb_q00/duckdb:parquet 1305328494 1181277024 1.11
fineweb_q01/duckdb:parquet 1436038416 1531131039 0.94
fineweb_q02/duckdb:parquet 1489239450 1418831831 1.05
fineweb_q03/duckdb:parquet 4404443050 4341656333 1.01
fineweb_q04/duckdb:parquet 2140116822 2114668512 1.01
fineweb_q05/duckdb:parquet 2517625469 2363525570 1.07
fineweb_q06/duckdb:parquet 4971906461 5053025289 0.98
fineweb_q07/duckdb:parquet 3083094796 3225519124 0.96
fineweb_q08/duckdb:parquet 1196703801 1360273040 0.88

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Benchmarks: Statistical and Population Genetics

Verdict: No clear signal (low confidence)
Attributed Vortex impact: +0.5%
Engines: DuckDB No clear signal (+0.5%, low confidence)
Vortex (geomean): 0.992x ➖
Parquet (geomean): 0.987x ➖
Shifts: Parquet (control) -1.3% · Median polish -1.1%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

duckdb / vortex-file-compressed (0.994x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
statpopgen_q00/duckdb:vortex-file-compressed 12325573 12206565 1.01
statpopgen_q01/duckdb:vortex-file-compressed 27434089 26487754 1.04
statpopgen_q02/duckdb:vortex-file-compressed 542140519 564302705 0.96
statpopgen_q03/duckdb:vortex-file-compressed 1042278688 1070106480 0.97
statpopgen_q04/duckdb:vortex-file-compressed 1056651294 1068553955 0.99
statpopgen_q05/duckdb:vortex-file-compressed 503745291 500957317 1.01
statpopgen_q06/duckdb:vortex-file-compressed 1537189526 1549907116 0.99
statpopgen_q07/duckdb:vortex-file-compressed 188386944 194728143 0.97
statpopgen_q08/duckdb:vortex-file-compressed 235472057 231776043 1.02
statpopgen_q09/duckdb:vortex-file-compressed 839553670 844379511 0.99
statpopgen_q10/duckdb:vortex-file-compressed 2572671042 2583136412 1.00
duckdb / vortex-compact (0.989x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
statpopgen_q00/duckdb:vortex-compact 11488314 11871869 0.97
statpopgen_q01/duckdb:vortex-compact 214431072 215744632 0.99
statpopgen_q02/duckdb:vortex-compact 564440886 566590591 1.00
statpopgen_q03/duckdb:vortex-compact 1161490381 1146513821 1.01
statpopgen_q04/duckdb:vortex-compact 1136266848 1169572379 0.97
statpopgen_q05/duckdb:vortex-compact 557786460 585434133 0.95
statpopgen_q06/duckdb:vortex-compact 1499261003 1500714778 1.00
statpopgen_q07/duckdb:vortex-compact 880764774 895793167 0.98
statpopgen_q08/duckdb:vortex-compact 915458749 924509669 0.99
statpopgen_q09/duckdb:vortex-compact 916319792 901021103 1.02
statpopgen_q10/duckdb:vortex-compact 2596355060 2609220457 1.00
duckdb / parquet (0.987x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
statpopgen_q00/duckdb:parquet 294642206 306818706 0.96
statpopgen_q01/duckdb:parquet 382103459 387465404 0.99
statpopgen_q02/duckdb:parquet 752099495 777639116 0.97
statpopgen_q03/duckdb:parquet 1201440016 1214277259 0.99
statpopgen_q04/duckdb:parquet 1197935245 1213284641 0.99
statpopgen_q05/duckdb:parquet 811131451 822339300 0.99
statpopgen_q06/duckdb:parquet 1434181389 1427069860 1.00
statpopgen_q07/duckdb:parquet 854160940 862684829 0.99
statpopgen_q08/duckdb:parquet 854811606 869340657 0.98
statpopgen_q09/duckdb:parquet 1023982922 1014272186 1.01
statpopgen_q10/duckdb:parquet 2211118603 2227364813 0.99

File Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
File Scale Format Base HEAD Change %
gnomad.genomes.v3.1.2.hgdp_tgp.chr21.vortex 100000 vortex-file-compressed 1.96 GB 1.96 GB 188.40 KB -0.0%

Totals:

  • vortex-compact: 959.62 MB → 959.62 MB (0.0%)
  • vortex-file-compressed: 1.96 GB → 1.96 GB (-0.0%)

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=10 on NVME

Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -3.4%
Engines: DataFusion No clear signal (-5.1%, environment too noisy confidence) · DuckDB No clear signal (-1.6%, low confidence)
Vortex (geomean): 1.112x ❌
Parquet (geomean): 1.113x ❌
Shifts: Parquet (control) +11.3% · Median polish +2.3%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.350x ❌, 0↑ 22↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-file-compressed 🚨 548888986 419489258 1.31
tpch_q02/datafusion:vortex-file-compressed 🚨 144825897 102222226 1.42
tpch_q03/datafusion:vortex-file-compressed 🚨 287121987 196664565 1.46
tpch_q04/datafusion:vortex-file-compressed 🚨 143267804 100866975 1.42
tpch_q05/datafusion:vortex-file-compressed 🚨 517415457 337625005 1.53
tpch_q06/datafusion:vortex-file-compressed 🚨 51204534 37008259 1.38
tpch_q07/datafusion:vortex-file-compressed 🚨 717201027 456792740 1.57
tpch_q08/datafusion:vortex-file-compressed 🚨 520968758 349797564 1.49
tpch_q09/datafusion:vortex-file-compressed 🚨 890393287 594949140 1.50
tpch_q10/datafusion:vortex-file-compressed 🚨 317748871 227696790 1.40
tpch_q11/datafusion:vortex-file-compressed 🚨 102617910 78056416 1.31
tpch_q12/datafusion:vortex-file-compressed 🚨 141700916 107711972 1.32
tpch_q13/datafusion:vortex-file-compressed 🚨 258227569 197966603 1.30
tpch_q14/datafusion:vortex-file-compressed 🚨 61966967 47423725 1.31
tpch_q15/datafusion:vortex-file-compressed 🚨 122667893 96415083 1.27
tpch_q16/datafusion:vortex-file-compressed 🚨 94524915 74168514 1.27
tpch_q17/datafusion:vortex-file-compressed 🚨 772818962 576935596 1.34
tpch_q18/datafusion:vortex-file-compressed 🚨 1086431316 824292656 1.32
tpch_q19/datafusion:vortex-file-compressed 🚨 75212295 64280278 1.17
tpch_q20/datafusion:vortex-file-compressed 🚨 197544888 157812282 1.25
tpch_q21/datafusion:vortex-file-compressed 🚨 783224141 609542991 1.28
tpch_q22/datafusion:vortex-file-compressed 🚨 63958086 55284300 1.16
datafusion / vortex-compact (1.109x ❌, 0↑ 10↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-compact 🚨 529970859 471892013 1.12
tpch_q02/datafusion:vortex-compact 🚨 123307884 107115595 1.15
tpch_q03/datafusion:vortex-compact 🚨 238279848 198785964 1.20
tpch_q04/datafusion:vortex-compact 🚨 126347759 107786744 1.17
tpch_q05/datafusion:vortex-compact 🚨 419337577 340589198 1.23
tpch_q06/datafusion:vortex-compact 63476007 57969360 1.09
tpch_q07/datafusion:vortex-compact 🚨 549191302 468403259 1.17
tpch_q08/datafusion:vortex-compact 🚨 397183209 349638718 1.14
tpch_q09/datafusion:vortex-compact 🚨 665951737 597794101 1.11
tpch_q10/datafusion:vortex-compact 267270897 243050581 1.10
tpch_q11/datafusion:vortex-compact 🚨 92373738 79507143 1.16
tpch_q12/datafusion:vortex-compact 165372851 150963872 1.10
tpch_q13/datafusion:vortex-compact 262771859 240533886 1.09
tpch_q14/datafusion:vortex-compact 70537134 64999273 1.09
tpch_q15/datafusion:vortex-compact 162237972 149388845 1.09
tpch_q16/datafusion:vortex-compact 86392474 79956701 1.08
tpch_q17/datafusion:vortex-compact 🚨 659554711 580109877 1.14
tpch_q18/datafusion:vortex-compact 892999084 826393192 1.08
tpch_q19/datafusion:vortex-compact 211906490 206672353 1.03
tpch_q20/datafusion:vortex-compact 186874626 181876703 1.03
tpch_q21/datafusion:vortex-compact 656263445 622708443 1.05
tpch_q22/datafusion:vortex-compact 64066820 63344793 1.01
datafusion / parquet (1.209x ❌, 0↑ 16↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
tpch_q01/datafusion:parquet 489939460 477667471 1.03
tpch_q02/datafusion:parquet 175466316 175132594 1.00
tpch_q03/datafusion:parquet 259495771 261591950 0.99
tpch_q04/datafusion:parquet 121786281 119782239 1.02
tpch_q05/datafusion:parquet 395997734 400829870 0.99
tpch_q06/datafusion:parquet 129110201 130629401 0.99
tpch_q07/datafusion:parquet 🚨 668249005 565625809 1.18
tpch_q08/datafusion:parquet 🚨 584467596 453219661 1.29
tpch_q09/datafusion:parquet 🚨 1003854773 728086399 1.38
tpch_q10/datafusion:parquet 🚨 810523226 576959694 1.40
tpch_q11/datafusion:parquet 🚨 162435277 117342913 1.38
tpch_q12/datafusion:parquet 🚨 256743682 210853185 1.22
tpch_q13/datafusion:parquet 🚨 402052961 347676339 1.16
tpch_q14/datafusion:parquet 🚨 195877115 153211004 1.28
tpch_q15/datafusion:parquet 🚨 338300460 260291747 1.30
tpch_q16/datafusion:parquet 🚨 145937011 122721882 1.19
tpch_q17/datafusion:parquet 🚨 936467234 663183342 1.41
tpch_q18/datafusion:parquet 🚨 1312020275 878103946 1.49
tpch_q19/datafusion:parquet 🚨 345433813 277344915 1.25
tpch_q20/datafusion:parquet 🚨 397245508 297445096 1.34
tpch_q21/datafusion:parquet 🚨 925562499 655578657 1.41
tpch_q22/datafusion:parquet 🚨 237438968 208243174 1.14
datafusion / arrow (1.008x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
tpch_q01/datafusion:arrow 696056842 694074641 1.00
tpch_q02/datafusion:arrow 112047223 113929943 0.98
tpch_q03/datafusion:arrow 506470320 500387210 1.01
tpch_q04/datafusion:arrow 366566545 360351633 1.02
tpch_q05/datafusion:arrow 745107243 735833091 1.01
tpch_q06/datafusion:arrow 325287269 327598340 0.99
tpch_q07/datafusion:arrow 1146849299 1136303316 1.01
tpch_q08/datafusion:arrow 953839485 936795387 1.02
tpch_q09/datafusion:arrow 1088971565 1094544130 0.99
tpch_q10/datafusion:arrow 632874909 623110652 1.02
tpch_q11/datafusion:arrow 92083265 91604988 1.01
tpch_q12/datafusion:arrow 1258917135 1162683333 1.08
tpch_q13/datafusion:arrow 473962030 464777357 1.02
tpch_q14/datafusion:arrow 359428789 344682439 1.04
tpch_q15/datafusion:arrow 736813188 750660399 0.98
tpch_q16/datafusion:arrow 81773882 79547034 1.03
tpch_q17/datafusion:arrow 954270821 957363999 1.00
tpch_q18/datafusion:arrow 1812380320 1842649715 0.98
tpch_q19/datafusion:arrow 550462540 550974759 1.00
tpch_q20/datafusion:arrow 506130620 506123343 1.00
tpch_q21/datafusion:arrow 3134692682 3122414231 1.00
tpch_q22/datafusion:arrow 80523719 81491163 0.99
duckdb / vortex-file-compressed (1.022x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-file-compressed 170936990 170480365 1.00
tpch_q02/duckdb:vortex-file-compressed 51170982 50753511 1.01
tpch_q03/duckdb:vortex-file-compressed 126662413 121610489 1.04
tpch_q04/duckdb:vortex-file-compressed 163956901 158644323 1.03
tpch_q05/duckdb:vortex-file-compressed 141598516 136801006 1.04
tpch_q06/duckdb:vortex-file-compressed 33603911 34491224 0.97
tpch_q07/duckdb:vortex-file-compressed 135222253 130852471 1.03
tpch_q08/duckdb:vortex-file-compressed 177514144 171576522 1.03
tpch_q09/duckdb:vortex-file-compressed 403788076 393266765 1.03
tpch_q10/duckdb:vortex-file-compressed 195696427 190817391 1.03
tpch_q11/duckdb:vortex-file-compressed 32833024 31526045 1.04
tpch_q12/duckdb:vortex-file-compressed 104743926 100058404 1.05
tpch_q13/duckdb:vortex-file-compressed 277375487 270804551 1.02
tpch_q14/duckdb:vortex-file-compressed 54136805 54108796 1.00
tpch_q15/duckdb:vortex-file-compressed 90337342 88379478 1.02
tpch_q16/duckdb:vortex-file-compressed 78272789 78115857 1.00
tpch_q17/duckdb:vortex-file-compressed 96141456 91829228 1.05
tpch_q18/duckdb:vortex-file-compressed 297896478 286679782 1.04
tpch_q19/duckdb:vortex-file-compressed 82304345 79503232 1.04
tpch_q20/duckdb:vortex-file-compressed 141758143 141327585 1.00
tpch_q21/duckdb:vortex-file-compressed 496896040 489021513 1.02
tpch_q22/duckdb:vortex-file-compressed 62978713 63865609 0.99
duckdb / vortex-compact (1.001x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-compact 229566250 228128034 1.01
tpch_q02/duckdb:vortex-compact 55969934 56787329 0.99
tpch_q03/duckdb:vortex-compact 136315293 137712062 0.99
tpch_q04/duckdb:vortex-compact 170299649 171267235 0.99
tpch_q05/duckdb:vortex-compact 164386538 165000878 1.00
tpch_q06/duckdb:vortex-compact 59176138 55364399 1.07
tpch_q07/duckdb:vortex-compact 173947593 172287450 1.01
tpch_q08/duckdb:vortex-compact 187984116 192753545 0.98
tpch_q09/duckdb:vortex-compact 427065215 424062503 1.01
tpch_q10/duckdb:vortex-compact 219475685 219675656 1.00
tpch_q11/duckdb:vortex-compact 38431091 38301854 1.00
tpch_q12/duckdb:vortex-compact 161735564 161183909 1.00
tpch_q13/duckdb:vortex-compact 322512670 322836812 1.00
tpch_q14/duckdb:vortex-compact 71895313 71557698 1.00
tpch_q15/duckdb:vortex-compact 114770639 113855821 1.01
tpch_q16/duckdb:vortex-compact 79908203 82184334 0.97
tpch_q17/duckdb:vortex-compact 107066128 106313588 1.01
tpch_q18/duckdb:vortex-compact 289776564 289430358 1.00
tpch_q19/duckdb:vortex-compact 95823923 96874415 0.99
tpch_q20/duckdb:vortex-compact 174721810 174754023 1.00
tpch_q21/duckdb:vortex-compact 520634988 513716517 1.01
tpch_q22/duckdb:vortex-compact 71766379 71625063 1.00
duckdb / parquet (1.024x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
tpch_q01/duckdb:parquet 265727045 268786987 0.99
tpch_q02/duckdb:parquet 107107435 98597157 1.09
tpch_q03/duckdb:parquet 214534385 211857769 1.01
tpch_q04/duckdb:parquet 140557974 134601791 1.04
tpch_q05/duckdb:parquet 234572426 226584015 1.04
tpch_q06/duckdb:parquet 75562348 75272239 1.00
tpch_q07/duckdb:parquet 189598339 184837177 1.03
tpch_q08/duckdb:parquet 267313831 261195227 1.02
tpch_q09/duckdb:parquet 482576058 472123132 1.02
tpch_q10/duckdb:parquet 628906343 621870392 1.01
tpch_q11/duckdb:parquet 65627595 62799353 1.05
tpch_q12/duckdb:parquet 134672398 130345690 1.03
tpch_q13/duckdb:parquet 446289542 429948656 1.04
tpch_q14/duckdb:parquet 181580793 178790561 1.02
tpch_q15/duckdb:parquet 105556318 102760653 1.03
tpch_q16/duckdb:parquet 169103846 162138109 1.04
tpch_q17/duckdb:parquet 184067157 180354622 1.02
tpch_q18/duckdb:parquet 370057626 365683202 1.01
tpch_q19/duckdb:parquet 289689502 282980822 1.02
tpch_q20/duckdb:parquet 225590584 231662556 0.97
tpch_q21/duckdb:parquet 568529067 554225353 1.03
tpch_q22/duckdb:parquet 296770136 291027800 1.02
duckdb / duckdb (1.001x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
tpch_q01/duckdb:duckdb 120304910 119932106 1.00
tpch_q02/duckdb:duckdb 49318487 51854152 0.95
tpch_q03/duckdb:duckdb 103116064 104118233 0.99
tpch_q04/duckdb:duckdb 138694193 139445208 0.99
tpch_q05/duckdb:duckdb 118900430 118965656 1.00
tpch_q06/duckdb:duckdb 45116285 44861847 1.01
tpch_q07/duckdb:duckdb 91053608 91018502 1.00
tpch_q08/duckdb:duckdb 117437435 119154323 0.99
tpch_q09/duckdb:duckdb 280133641 280326722 1.00
tpch_q10/duckdb:duckdb 211916279 211905652 1.00
tpch_q11/duckdb:duckdb 19322480 19233046 1.00
tpch_q12/duckdb:duckdb 91279495 90208164 1.01
tpch_q13/duckdb:duckdb 232636272 226897169 1.03
tpch_q14/duckdb:duckdb 78499802 77982902 1.01
tpch_q15/duckdb:duckdb 81456236 81797874 1.00
tpch_q16/duckdb:duckdb 78082296 76446061 1.02
tpch_q17/duckdb:duckdb 87761984 88547396 0.99
tpch_q18/duckdb:duckdb 234032652 235731145 0.99
tpch_q19/duckdb:duckdb 124010729 123890492 1.00
tpch_q20/duckdb:duckdb 117906732 117914657 1.00
tpch_q21/duckdb:duckdb 302476005 300776020 1.01
tpch_q22/duckdb:duckdb 72049940 70062411 1.03

File Size Changes (26 files changed, +0.0% overall, 14↑ 12↓)
File Scale Format Base HEAD Change %
partsupp_0.vortex 10.0 vortex-compact 104.52 MB 105.30 MB +807.48 KB +0.8%
orders_2.vortex 10.0 vortex-file-compressed 145.29 MB 146.25 MB +980.86 KB +0.7%
part_1.vortex 10.0 vortex-file-compressed 24.73 MB 24.86 MB +127.91 KB +0.5%
supplier_0.vortex 10.0 vortex-file-compressed 5.72 MB 5.75 MB +25.98 KB +0.4%
part_0.vortex 10.0 vortex-compact 16.86 MB 16.94 MB +75.85 KB +0.4%
lineitem_3.vortex 10.0 vortex-file-compressed 133.08 MB 133.38 MB +311.73 KB +0.2%
lineitem_4.vortex 10.0 vortex-file-compressed 134.01 MB 134.09 MB +83.27 KB +0.1%
lineitem_1.vortex 10.0 vortex-file-compressed 133.69 MB 133.76 MB +68.07 KB +0.0%
lineitem_2.vortex 10.0 vortex-file-compressed 133.95 MB 134.00 MB +48.05 KB +0.0%
orders_1.vortex 10.0 vortex-file-compressed 145.63 MB 145.68 MB +50.27 KB +0.0%
lineitem_0.vortex 10.0 vortex-file-compressed 134.29 MB 134.32 MB +26.73 KB +0.0%
partsupp_0.vortex 10.0 vortex-file-compressed 120.32 MB 120.34 MB +23.42 KB +0.0%
lineitem_8.vortex 10.0 vortex-file-compressed 133.49 MB 133.50 MB +10.91 KB +0.0%
partsupp_1.vortex 10.0 vortex-file-compressed 120.30 MB 120.30 MB +3.07 KB +0.0%
orders_0.vortex 10.0 vortex-file-compressed 144.41 MB 144.39 MB 19.80 KB -0.0%
lineitem_6.vortex 10.0 vortex-file-compressed 132.72 MB 132.70 MB 20.02 KB -0.0%
lineitem_7.vortex 10.0 vortex-file-compressed 133.62 MB 133.58 MB 38.68 KB -0.0%
lineitem_10.vortex 10.0 vortex-file-compressed 134.22 MB 134.17 MB 45.89 KB -0.0%
customer_0.vortex 10.0 vortex-file-compressed 88.52 MB 88.49 MB 38.92 KB -0.0%
lineitem_9.vortex 10.0 vortex-file-compressed 134.24 MB 134.14 MB 102.66 KB -0.1%
lineitem_12.vortex 10.0 vortex-file-compressed 134.10 MB 133.97 MB 128.48 KB -0.1%
lineitem_5.vortex 10.0 vortex-file-compressed 133.65 MB 133.50 MB 146.88 KB -0.1%
lineitem_11.vortex 10.0 vortex-file-compressed 134.46 MB 134.16 MB 315.25 KB -0.2%
part_0.vortex 10.0 vortex-file-compressed 24.91 MB 24.82 MB 92.06 KB -0.4%
partsupp_1.vortex 10.0 vortex-compact 105.26 MB 104.58 MB 697.08 KB -0.6%
part_1.vortex 10.0 vortex-compact 17.10 MB 16.86 MB 241.63 KB -1.4%

Totals:

  • vortex-compact: 1.93 GB → 1.93 GB (-0.0%)
  • vortex-file-compressed: 2.50 GB → 2.50 GB (+0.0%)

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Benchmarks: Clickbench on NVME

Verdict: No clear signal (low confidence)
Attributed Vortex impact: -1.2%
Engines: DataFusion No clear signal (-1.2%, environment too noisy confidence) · DuckDB No clear signal (-0.7%, low confidence)
Vortex (geomean): 1.019x ➖
Parquet (geomean): 1.027x ➖
Shifts: Parquet (control) +2.7% · Median polish +1.9%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.026x ➖, 0↑ 1↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
clickbench_q00/datafusion:vortex-file-compressed 🚨 1960237 1605609 1.22
clickbench_q01/datafusion:vortex-file-compressed 15998747 16177677 0.99
clickbench_q02/datafusion:vortex-file-compressed 35748609 35116300 1.02
clickbench_q03/datafusion:vortex-file-compressed 39635208 38404956 1.03
clickbench_q04/datafusion:vortex-file-compressed 239996008 230625464 1.04
clickbench_q05/datafusion:vortex-file-compressed 312933684 305322056 1.02
clickbench_q06/datafusion:vortex-file-compressed 1691514 1586335 1.07
clickbench_q07/datafusion:vortex-file-compressed 19192243 19190428 1.00
clickbench_q08/datafusion:vortex-file-compressed 341140744 330159329 1.03
clickbench_q09/datafusion:vortex-file-compressed 464456319 447540200 1.04
clickbench_q10/datafusion:vortex-file-compressed 69528930 68764742 1.01
clickbench_q11/datafusion:vortex-file-compressed 79770992 81326682 0.98
clickbench_q12/datafusion:vortex-file-compressed 263822869 258605907 1.02
clickbench_q13/datafusion:vortex-file-compressed 434643678 406186926 1.07
clickbench_q14/datafusion:vortex-file-compressed 256908225 259066722 0.99
clickbench_q15/datafusion:vortex-file-compressed 280320543 278233411 1.01
clickbench_q16/datafusion:vortex-file-compressed 680335777 648194777 1.05
clickbench_q17/datafusion:vortex-file-compressed 666103464 637909698 1.04
clickbench_q18/datafusion:vortex-file-compressed 1394089994 1370612517 1.02
clickbench_q19/datafusion:vortex-file-compressed 28437221 28212669 1.01
clickbench_q20/datafusion:vortex-file-compressed 334303948 326490327 1.02
clickbench_q21/datafusion:vortex-file-compressed 412881262 404240904 1.02
clickbench_q22/datafusion:vortex-file-compressed 511070564 493154980 1.04
clickbench_q23/datafusion:vortex-file-compressed 554868216 562985421 0.99
clickbench_q24/datafusion:vortex-file-compressed 43512921 45170460 0.96
clickbench_q25/datafusion:vortex-file-compressed 76240287 76171312 1.00
clickbench_q26/datafusion:vortex-file-compressed 41309981 40169576 1.03
clickbench_q27/datafusion:vortex-file-compressed 426125052 419820697 1.02
clickbench_q28/datafusion:vortex-file-compressed 2414661323 2390955450 1.01
clickbench_q29/datafusion:vortex-file-compressed 50813829 54791053 0.93
clickbench_q30/datafusion:vortex-file-compressed 250791075 245612670 1.02
clickbench_q31/datafusion:vortex-file-compressed 271324533 258230596 1.05
clickbench_q32/datafusion:vortex-file-compressed 1086446501 1023971751 1.06
clickbench_q33/datafusion:vortex-file-compressed 1458608813 1399512241 1.04
clickbench_q34/datafusion:vortex-file-compressed 1448684885 1396726710 1.04
clickbench_q35/datafusion:vortex-file-compressed 249486551 248927303 1.00
clickbench_q36/datafusion:vortex-file-compressed 62108869 60188939 1.03
clickbench_q37/datafusion:vortex-file-compressed 26598504 25730069 1.03
clickbench_q38/datafusion:vortex-file-compressed 16260269 16556808 0.98
clickbench_q39/datafusion:vortex-file-compressed 125212902 120271225 1.04
clickbench_q40/datafusion:vortex-file-compressed 12858324 12237728 1.05
clickbench_q41/datafusion:vortex-file-compressed 13371809 12331550 1.08
clickbench_q42/datafusion:vortex-file-compressed 12245595 11648359 1.05
datafusion / parquet (1.039x ➖, 0↑ 3↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
clickbench_q00/datafusion:parquet 🚨 1968305 1556078 1.26
clickbench_q01/datafusion:parquet 20551002 19608993 1.05
clickbench_q02/datafusion:parquet 44063314 44850790 0.98
clickbench_q03/datafusion:parquet 34789564 36343190 0.96
clickbench_q04/datafusion:parquet 281576939 270725793 1.04
clickbench_q05/datafusion:parquet 🚨 348781272 314095263 1.11
clickbench_q06/datafusion:parquet 1548638 1473818 1.05
clickbench_q07/datafusion:parquet 22027065 21558662 1.02
clickbench_q08/datafusion:parquet 343455691 330361493 1.04
clickbench_q09/datafusion:parquet 478920991 479584353 1.00
clickbench_q10/datafusion:parquet 97689377 93959270 1.04
clickbench_q11/datafusion:parquet 121195698 118543030 1.02
clickbench_q12/datafusion:parquet 321718512 309532235 1.04
clickbench_q13/datafusion:parquet 487072952 467110811 1.04
clickbench_q14/datafusion:parquet 319698860 316301382 1.01
clickbench_q15/datafusion:parquet 283279975 277305838 1.02
clickbench_q16/datafusion:parquet 689309463 643129019 1.07
clickbench_q17/datafusion:parquet 667780474 648029474 1.03
clickbench_q18/datafusion:parquet 1476120367 1355917642 1.09
clickbench_q19/datafusion:parquet 🚨 35434878 25977285 1.36
clickbench_q20/datafusion:parquet 570638647 575006779 0.99
clickbench_q21/datafusion:parquet 636857992 628331874 1.01
clickbench_q22/datafusion:parquet 927834745 917735947 1.01
clickbench_q23/datafusion:parquet 4210231198 4037496166 1.04
clickbench_q24/datafusion:parquet 54858316 53008877 1.03
clickbench_q25/datafusion:parquet 130685248 127683675 1.02
clickbench_q26/datafusion:parquet 53296718 52399671 1.02
clickbench_q27/datafusion:parquet 665768179 654889382 1.02
clickbench_q28/datafusion:parquet 2472620344 2433944473 1.02
clickbench_q29/datafusion:parquet 52077402 53097156 0.98
clickbench_q30/datafusion:parquet 321366052 316047548 1.02
clickbench_q31/datafusion:parquet 352779502 345532827 1.02
clickbench_q32/datafusion:parquet 1108031468 1088507512 1.02
clickbench_q33/datafusion:parquet 1531748929 1492964526 1.03
clickbench_q34/datafusion:parquet 1531807891 1506309155 1.02
clickbench_q35/datafusion:parquet 253775597 248959123 1.02
clickbench_q36/datafusion:parquet 110024150 108877439 1.01
clickbench_q37/datafusion:parquet 44517406 43490500 1.02
clickbench_q38/datafusion:parquet 63214016 61563897 1.03
clickbench_q39/datafusion:parquet 218321570 208373376 1.05
clickbench_q40/datafusion:parquet 24385651 22591238 1.08
clickbench_q41/datafusion:parquet 22336060 22194239 1.01
clickbench_q42/datafusion:parquet 22971566 22110216 1.04
duckdb / vortex-file-compressed (1.012x ➖, 3↑ 1↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
clickbench_q00/duckdb:vortex-file-compressed 6794564 6949585 0.98
clickbench_q01/duckdb:vortex-file-compressed 11114633 10823945 1.03
clickbench_q02/duckdb:vortex-file-compressed 27020466 26139795 1.03
clickbench_q03/duckdb:vortex-file-compressed 30863992 31615330 0.98
clickbench_q04/duckdb:vortex-file-compressed 192182667 188142053 1.02
clickbench_q05/duckdb:vortex-file-compressed 186072755 176289744 1.06
clickbench_q06/duckdb:vortex-file-compressed 20481047 20257044 1.01
clickbench_q07/duckdb:vortex-file-compressed 13958853 14023835 1.00
clickbench_q08/duckdb:vortex-file-compressed 268631549 265007569 1.01
clickbench_q09/duckdb:vortex-file-compressed 347394045 342513159 1.01
clickbench_q10/duckdb:vortex-file-compressed 71120186 67916021 1.05
clickbench_q11/duckdb:vortex-file-compressed 79761396 79523031 1.00
clickbench_q12/duckdb:vortex-file-compressed 203506345 198246865 1.03
clickbench_q13/duckdb:vortex-file-compressed 440992075 414021544 1.07
clickbench_q14/duckdb:vortex-file-compressed 239324137 233164087 1.03
clickbench_q15/duckdb:vortex-file-compressed 249096111 243560689 1.02
clickbench_q16/duckdb:vortex-file-compressed 551642478 533492404 1.03
clickbench_q17/duckdb:vortex-file-compressed 437013687 431022192 1.01
clickbench_q18/duckdb:vortex-file-compressed 973992076 949957034 1.03
clickbench_q19/duckdb:vortex-file-compressed 21237792 22310521 0.95
clickbench_q20/duckdb:vortex-file-compressed 315295280 310569444 1.02
clickbench_q21/duckdb:vortex-file-compressed 427075735 394929818 1.08
clickbench_q22/duckdb:vortex-file-compressed 🚀 571726201 650884253 0.88
clickbench_q23/duckdb:vortex-file-compressed 206347717 200635094 1.03
clickbench_q24/duckdb:vortex-file-compressed 🚀 43452625 49288702 0.88
clickbench_q25/duckdb:vortex-file-compressed 82596799 84381476 0.98
clickbench_q26/duckdb:vortex-file-compressed 54224180 49642837 1.09
clickbench_q27/duckdb:vortex-file-compressed 419930827 414402778 1.01
clickbench_q28/duckdb:vortex-file-compressed 3023713179 3057414077 0.99
clickbench_q29/duckdb:vortex-file-compressed 🚀 29711151 37010179 0.80
clickbench_q30/duckdb:vortex-file-compressed 220545805 217693375 1.01
clickbench_q31/duckdb:vortex-file-compressed 317288146 320978572 0.99
clickbench_q32/duckdb:vortex-file-compressed 1144088291 1133284981 1.01
clickbench_q33/duckdb:vortex-file-compressed 1134214173 1143528850 0.99
clickbench_q34/duckdb:vortex-file-compressed 1247729865 1213944669 1.03
clickbench_q35/duckdb:vortex-file-compressed 379240395 375493465 1.01
clickbench_q36/duckdb:vortex-file-compressed 28091822 27476145 1.02
clickbench_q37/duckdb:vortex-file-compressed 20847588 19482056 1.07
clickbench_q38/duckdb:vortex-file-compressed 22532067 21387317 1.05
clickbench_q39/duckdb:vortex-file-compressed 46137821 44345996 1.04
clickbench_q40/duckdb:vortex-file-compressed 20216950 19046233 1.06
clickbench_q41/duckdb:vortex-file-compressed 19876926 18545572 1.07
clickbench_q42/duckdb:vortex-file-compressed 🚨 23523502 21003873 1.12
duckdb / parquet (1.016x ➖, 0↑ 2↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
clickbench_q00/duckdb:parquet 22493796 22627876 0.99
clickbench_q01/duckdb:parquet 30787574 29270868 1.05
clickbench_q02/duckdb:parquet 50071133 51699919 0.97
clickbench_q03/duckdb:parquet 40967558 41587282 0.99
clickbench_q04/duckdb:parquet 211805154 204220790 1.04
clickbench_q05/duckdb:parquet 263693410 260579248 1.01
clickbench_q06/duckdb:parquet 47734376 48011735 0.99
clickbench_q07/duckdb:parquet 32070686 31237662 1.03
clickbench_q08/duckdb:parquet 277955214 272316468 1.02
clickbench_q09/duckdb:parquet 405172057 403106575 1.01
clickbench_q10/duckdb:parquet 84548871 83925484 1.01
clickbench_q11/duckdb:parquet 101141827 99689604 1.01
clickbench_q12/duckdb:parquet 287094861 284818951 1.01
clickbench_q13/duckdb:parquet 488788720 471082724 1.04
clickbench_q14/duckdb:parquet 319783589 315828550 1.01
clickbench_q15/duckdb:parquet 262425735 263975437 0.99
clickbench_q16/duckdb:parquet 609391824 598848529 1.02
clickbench_q17/duckdb:parquet 506365927 498419335 1.02
clickbench_q18/duckdb:parquet 1060314905 1043648463 1.02
clickbench_q19/duckdb:parquet 27622121 27328381 1.01
clickbench_q20/duckdb:parquet 415877333 417089329 1.00
clickbench_q21/duckdb:parquet 534583704 533781470 1.00
clickbench_q22/duckdb:parquet 928203949 921663136 1.01
clickbench_q23/duckdb:parquet 269055454 278507491 0.97
clickbench_q24/duckdb:parquet 72049409 73369560 0.98
clickbench_q25/duckdb:parquet 164754984 164526806 1.00
clickbench_q26/duckdb:parquet 55391235 52022841 1.06
clickbench_q27/duckdb:parquet 471687906 474330362 0.99
clickbench_q28/duckdb:parquet 4821747648 4767065185 1.01
clickbench_q29/duckdb:parquet 43206641 42521177 1.02
clickbench_q30/duckdb:parquet 315554044 311079414 1.01
clickbench_q31/duckdb:parquet 381594456 374860221 1.02
clickbench_q32/duckdb:parquet 1135219985 1102692405 1.03
clickbench_q33/duckdb:parquet 1131228947 1108060487 1.02
clickbench_q34/duckdb:parquet 1178066635 1157146007 1.02
clickbench_q35/duckdb:parquet 371148753 370922772 1.00
clickbench_q36/duckdb:parquet 45930163 45765277 1.00
clickbench_q37/duckdb:parquet 34361498 34465393 1.00
clickbench_q38/duckdb:parquet 35947771 34415452 1.04
clickbench_q39/duckdb:parquet 🚨 89632853 78443038 1.14
clickbench_q40/duckdb:parquet 20922259 20898053 1.00
clickbench_q41/duckdb:parquet 21290276 20268513 1.05
clickbench_q42/duckdb:parquet 🚨 24540460 22125837 1.11
duckdb / duckdb (1.007x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
clickbench_q00/duckdb:duckdb 17351032 16990263 1.02
clickbench_q01/duckdb:duckdb 34511415 34158357 1.01
clickbench_q02/duckdb:duckdb 51514567 51171948 1.01
clickbench_q03/duckdb:duckdb 55663371 55853461 1.00
clickbench_q04/duckdb:duckdb 211914991 211822678 1.00
clickbench_q05/duckdb:duckdb 284755626 284733259 1.00
clickbench_q06/duckdb:duckdb 35136424 33728037 1.04
clickbench_q07/duckdb:duckdb 35261600 35504479 0.99
clickbench_q08/duckdb:duckdb 276254034 275918542 1.00
clickbench_q09/duckdb:duckdb 400144035 395969418 1.01
clickbench_q10/duckdb:duckdb 118522174 118348936 1.00
clickbench_q11/duckdb:duckdb 128653214 126821395 1.01
clickbench_q12/duckdb:duckdb 264054107 258794120 1.02
clickbench_q13/duckdb:duckdb 447206093 443226112 1.01
clickbench_q14/duckdb:duckdb 288875683 287459038 1.00
clickbench_q15/duckdb:duckdb 245860785 244307890 1.01
clickbench_q16/duckdb:duckdb 572875361 562637018 1.02
clickbench_q17/duckdb:duckdb 481548056 471804930 1.02
clickbench_q18/duckdb:duckdb 1027512399 1011948044 1.02
clickbench_q19/duckdb:duckdb 35541410 34886460 1.02
clickbench_q20/duckdb:duckdb 477399198 491524053 0.97
clickbench_q21/duckdb:duckdb 487500691 502374393 0.97
clickbench_q22/duckdb:duckdb 536214051 526008724 1.02
clickbench_q23/duckdb:duckdb 257941399 254273032 1.01
clickbench_q24/duckdb:duckdb 62724406 61822504 1.01
clickbench_q25/duckdb:duckdb 149862103 149154064 1.00
clickbench_q26/duckdb:duckdb 62807973 61540832 1.02
clickbench_q27/duckdb:duckdb 425431179 426654391 1.00
clickbench_q28/duckdb:duckdb 4413414993 4426933911 1.00
clickbench_q29/duckdb:duckdb 50620476 50710392 1.00
clickbench_q30/duckdb:duckdb 282969557 280615605 1.01
clickbench_q31/duckdb:duckdb 366807959 364706656 1.01
clickbench_q32/duckdb:duckdb 1136711283 1116341584 1.02
clickbench_q33/duckdb:duckdb 1158141552 1132024759 1.02
clickbench_q34/duckdb:duckdb 1241666485 1206944888 1.03
clickbench_q35/duckdb:duckdb 303140132 299148713 1.01
clickbench_q36/duckdb:duckdb 48626971 53832145 0.90
clickbench_q37/duckdb:duckdb 29030133 28500612 1.02
clickbench_q38/duckdb:duckdb 33041994 34356945 0.96
clickbench_q39/duckdb:duckdb 94948206 87742870 1.08
clickbench_q40/duckdb:duckdb 27853228 27415870 1.02
clickbench_q41/duckdb:duckdb 27089153 27130434 1.00
clickbench_q42/duckdb:duckdb 29449257 28867665 1.02

File Size Changes (104 files changed, +0.0% overall, 55↑ 49↓)
File Scale Format Base HEAD Change %
hits_55.vortex 1.0 vortex-compact 94.94 MB 96.04 MB +1.09 MB +1.2%
hits_69.vortex 1.0 vortex-file-compressed 122.95 MB 123.35 MB +417.64 KB +0.3%
hits_21.vortex 1.0 vortex-file-compressed 94.01 MB 94.26 MB +257.30 KB +0.3%
hits_65.vortex 1.0 vortex-compact 129.39 MB 129.71 MB +327.21 KB +0.2%
hits_97.vortex 1.0 vortex-file-compressed 106.98 MB 107.24 MB +262.10 KB +0.2%
hits_74.vortex 1.0 vortex-file-compressed 119.89 MB 120.16 MB +282.12 KB +0.2%
hits_10.vortex 1.0 vortex-file-compressed 69.68 MB 69.81 MB +137.11 KB +0.2%
hits_4.vortex 1.0 vortex-file-compressed 108.70 MB 108.91 MB +210.91 KB +0.2%
hits_66.vortex 1.0 vortex-file-compressed 90.19 MB 90.34 MB +147.80 KB +0.2%
hits_3.vortex 1.0 vortex-file-compressed 142.30 MB 142.49 MB +199.55 KB +0.1%
hits_40.vortex 1.0 vortex-file-compressed 117.81 MB 117.93 MB +125.82 KB +0.1%
hits_53.vortex 1.0 vortex-file-compressed 87.47 MB 87.56 MB +91.66 KB +0.1%
hits_81.vortex 1.0 vortex-file-compressed 100.84 MB 100.94 MB +102.58 KB +0.1%
hits_19.vortex 1.0 vortex-file-compressed 73.52 MB 73.60 MB +73.48 KB +0.1%
hits_88.vortex 1.0 vortex-file-compressed 111.39 MB 111.49 MB +103.02 KB +0.1%
hits_32.vortex 1.0 vortex-file-compressed 66.84 MB 66.90 MB +57.86 KB +0.1%
hits_76.vortex 1.0 vortex-file-compressed 114.31 MB 114.40 MB +95.17 KB +0.1%
hits_57.vortex 1.0 vortex-file-compressed 128.21 MB 128.30 MB +95.30 KB +0.1%
hits_5.vortex 1.0 vortex-file-compressed 92.91 MB 92.98 MB +68.88 KB +0.1%
hits_28.vortex 1.0 vortex-file-compressed 119.75 MB 119.83 MB +87.27 KB +0.1%
hits_62.vortex 1.0 vortex-file-compressed 117.34 MB 117.42 MB +82.98 KB +0.1%
hits_89.vortex 1.0 vortex-file-compressed 184.50 MB 184.62 MB +123.97 KB +0.1%
hits_0.vortex 1.0 vortex-file-compressed 89.54 MB 89.60 MB +57.37 KB +0.1%
hits_58.vortex 1.0 vortex-file-compressed 90.43 MB 90.49 MB +55.93 KB +0.1%
hits_33.vortex 1.0 vortex-file-compressed 57.27 MB 57.30 MB +34.89 KB +0.1%
hits_51.vortex 1.0 vortex-file-compressed 277.65 MB 277.81 MB +168.44 KB +0.1%
hits_35.vortex 1.0 vortex-file-compressed 115.11 MB 115.18 MB +69.76 KB +0.1%
hits_16.vortex 1.0 vortex-file-compressed 79.93 MB 79.97 MB +41.91 KB +0.1%
hits_7.vortex 1.0 vortex-file-compressed 93.96 MB 94.00 MB +44.92 KB +0.0%
hits_24.vortex 1.0 vortex-file-compressed 77.44 MB 77.47 MB +36.26 KB +0.0%
hits_38.vortex 1.0 vortex-file-compressed 99.26 MB 99.30 MB +45.25 KB +0.0%
hits_30.vortex 1.0 vortex-file-compressed 87.14 MB 87.18 MB +38.84 KB +0.0%
hits_95.vortex 1.0 vortex-file-compressed 96.57 MB 96.62 MB +42.92 KB +0.0%
hits_65.vortex 1.0 vortex-file-compressed 183.79 MB 183.86 MB +79.23 KB +0.0%
hits_93.vortex 1.0 vortex-file-compressed 90.61 MB 90.64 MB +36.40 KB +0.0%
hits_2.vortex 1.0 vortex-file-compressed 186.45 MB 186.53 MB +74.70 KB +0.0%
hits_20.vortex 1.0 vortex-file-compressed 62.95 MB 62.97 MB +22.34 KB +0.0%
hits_85.vortex 1.0 vortex-file-compressed 91.89 MB 91.92 MB +31.50 KB +0.0%
hits_17.vortex 1.0 vortex-file-compressed 87.47 MB 87.50 MB +29.45 KB +0.0%
hits_22.vortex 1.0 vortex-file-compressed 78.58 MB 78.61 MB +21.78 KB +0.0%
hits_96.vortex 1.0 vortex-file-compressed 135.68 MB 135.71 MB +33.41 KB +0.0%
hits_99.vortex 1.0 vortex-file-compressed 123.15 MB 123.18 MB +25.20 KB +0.0%
hits_55.vortex 1.0 vortex-file-compressed 169.25 MB 169.28 MB +33.20 KB +0.0%
hits_82.vortex 1.0 vortex-file-compressed 99.85 MB 99.87 MB +19.56 KB +0.0%
hits_44.vortex 1.0 vortex-file-compressed 186.89 MB 186.93 MB +34.57 KB +0.0%
hits_56.vortex 1.0 vortex-file-compressed 123.55 MB 123.57 MB +17.73 KB +0.0%
hits_70.vortex 1.0 vortex-file-compressed 93.63 MB 93.64 MB +10.25 KB +0.0%
hits_60.vortex 1.0 vortex-file-compressed 103.41 MB 103.42 MB +10.88 KB +0.0%
hits_78.vortex 1.0 vortex-file-compressed 164.20 MB 164.22 MB +15.71 KB +0.0%
hits_23.vortex 1.0 vortex-file-compressed 77.92 MB 77.93 MB +6.08 KB +0.0%
hits_8.vortex 1.0 vortex-file-compressed 93.27 MB 93.28 MB +6.55 KB +0.0%
hits_47.vortex 1.0 vortex-file-compressed 41.32 MB 41.32 MB +1.95 KB +0.0%
hits_64.vortex 1.0 vortex-file-compressed 81.23 MB 81.23 MB +2.18 KB +0.0%
hits_37.vortex 1.0 vortex-file-compressed 85.83 MB 85.84 MB +2.26 KB +0.0%
hits_68.vortex 1.0 vortex-file-compressed 123.16 MB 123.17 MB +704 B +0.0%
hits_13.vortex 1.0 vortex-file-compressed 99.26 MB 99.26 MB 472 B -0.0%
hits_61.vortex 1.0 vortex-file-compressed 101.29 MB 101.29 MB 1.80 KB -0.0%
hits_42.vortex 1.0 vortex-file-compressed 223.17 MB 223.16 MB 8.73 KB -0.0%
hits_72.vortex 1.0 vortex-file-compressed 84.84 MB 84.83 MB 3.83 KB -0.0%
hits_11.vortex 1.0 vortex-file-compressed 79.99 MB 79.99 MB 7.90 KB -0.0%
hits_77.vortex 1.0 vortex-file-compressed 168.69 MB 168.67 MB 19.99 KB -0.0%
hits_15.vortex 1.0 vortex-file-compressed 89.32 MB 89.31 MB 12.63 KB -0.0%
hits_73.vortex 1.0 vortex-file-compressed 110.55 MB 110.53 MB 15.91 KB -0.0%
hits_6.vortex 1.0 vortex-file-compressed 93.35 MB 93.34 MB 17.21 KB -0.0%
hits_50.vortex 1.0 vortex-file-compressed 179.38 MB 179.34 MB 44.67 KB -0.0%
hits_31.vortex 1.0 vortex-file-compressed 90.10 MB 90.07 MB 24.60 KB -0.0%
hits_9.vortex 1.0 vortex-file-compressed 99.25 MB 99.22 MB 27.60 KB -0.0%
hits_90.vortex 1.0 vortex-file-compressed 142.11 MB 142.07 MB 46.47 KB -0.0%
hits_45.vortex 1.0 vortex-file-compressed 122.16 MB 122.12 MB 42.38 KB -0.0%
hits_18.vortex 1.0 vortex-file-compressed 104.86 MB 104.83 MB 37.05 KB -0.0%
hits_12.vortex 1.0 vortex-file-compressed 101.07 MB 101.03 MB 39.61 KB -0.0%
hits_92.vortex 1.0 vortex-file-compressed 146.96 MB 146.90 MB 57.62 KB -0.0%
hits_34.vortex 1.0 vortex-file-compressed 97.84 MB 97.80 MB 41.96 KB -0.0%
hits_29.vortex 1.0 vortex-file-compressed 59.81 MB 59.79 MB 26.25 KB -0.0%
hits_94.vortex 1.0 vortex-file-compressed 138.89 MB 138.82 MB 64.80 KB -0.0%
hits_39.vortex 1.0 vortex-file-compressed 80.40 MB 80.36 MB 38.23 KB -0.0%
hits_54.vortex 1.0 vortex-file-compressed 221.48 MB 221.36 MB 120.64 KB -0.1%
hits_27.vortex 1.0 vortex-file-compressed 122.37 MB 122.30 MB 66.72 KB -0.1%
hits_98.vortex 1.0 vortex-file-compressed 118.42 MB 118.36 MB 65.51 KB -0.1%
hits_59.vortex 1.0 vortex-file-compressed 101.92 MB 101.86 MB 56.84 KB -0.1%
hits_46.vortex 1.0 vortex-file-compressed 69.48 MB 69.44 MB 38.95 KB -0.1%
hits_26.vortex 1.0 vortex-file-compressed 110.13 MB 110.06 MB 64.29 KB -0.1%
hits_41.vortex 1.0 vortex-file-compressed 224.31 MB 224.18 MB 132.05 KB -0.1%
hits_80.vortex 1.0 vortex-file-compressed 105.06 MB 104.99 MB 70.05 KB -0.1%
hits_43.vortex 1.0 vortex-file-compressed 227.46 MB 227.31 MB 154.10 KB -0.1%
hits_86.vortex 1.0 vortex-file-compressed 69.38 MB 69.33 MB 48.06 KB -0.1%
hits_71.vortex 1.0 vortex-file-compressed 102.08 MB 102.01 MB 73.59 KB -0.1%
hits_48.vortex 1.0 vortex-file-compressed 28.05 MB 28.03 MB 20.52 KB -0.1%
hits_87.vortex 1.0 vortex-file-compressed 172.61 MB 172.48 MB 129.43 KB -0.1%
hits_63.vortex 1.0 vortex-file-compressed 69.32 MB 69.27 MB 54.41 KB -0.1%
hits_25.vortex 1.0 vortex-file-compressed 114.05 MB 113.92 MB 124.95 KB -0.1%
hits_67.vortex 1.0 vortex-file-compressed 184.24 MB 184.04 MB 207.56 KB -0.1%
hits_75.vortex 1.0 vortex-file-compressed 63.33 MB 63.26 MB 71.47 KB -0.1%
hits_36.vortex 1.0 vortex-file-compressed 68.64 MB 68.56 MB 80.67 KB -0.1%
hits_14.vortex 1.0 vortex-file-compressed 111.40 MB 111.26 MB 147.86 KB -0.1%
hits_49.vortex 1.0 vortex-file-compressed 75.92 MB 75.81 MB 109.48 KB -0.1%
hits_91.vortex 1.0 vortex-file-compressed 97.26 MB 97.12 MB 145.50 KB -0.1%
hits_1.vortex 1.0 vortex-file-compressed 138.68 MB 138.46 MB 229.62 KB -0.2%
hits_83.vortex 1.0 vortex-file-compressed 90.01 MB 89.85 MB 163.59 KB -0.2%
hits_79.vortex 1.0 vortex-file-compressed 145.33 MB 145.07 MB 266.81 KB -0.2%
hits_27.vortex 1.0 vortex-compact 69.95 MB 69.81 MB 143.06 KB -0.2%
hits_84.vortex 1.0 vortex-file-compressed 117.60 MB 117.30 MB 306.29 KB -0.3%
hits_52.vortex 1.0 vortex-file-compressed 105.38 MB 105.07 MB 312.57 KB -0.3%
duckdb.db 1.0 vortex-compact 268.00 KB 0 B 268.00 KB -100.0%

Totals:

  • vortex-compact: 7.04 GB → 7.05 GB (+0.0%)
  • vortex-file-compressed: 11.02 GB → 11.02 GB (+0.0%)

Signed-off-by: Adam Gutglick <adam@spiraldb.com>
@AdamGS AdamGS added the action/benchmark Trigger full benchmarks to run on this PR label Jun 10, 2026
@github-actions github-actions Bot removed the action/benchmark Trigger full benchmarks to run on this PR label Jun 10, 2026
@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=1 on S3

Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: +11.2%
Engines: DataFusion No clear signal (+29.0%, environment too noisy confidence) · DuckDB No clear signal (-4.1%, environment too noisy confidence)
Vortex (geomean): 1.125x ➖
Parquet (geomean): 1.012x ➖
Shifts: Parquet (control) +1.2% · Median polish +5.0%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.330x ❌, 0↑ 13↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-file-compressed 272960573 331602657 0.82
tpch_q02/datafusion:vortex-file-compressed 470616173 526176635 0.89
tpch_q03/datafusion:vortex-file-compressed 451535931 557017885 0.81
tpch_q04/datafusion:vortex-file-compressed 🚨 443265344 280776663 1.58
tpch_q05/datafusion:vortex-file-compressed 🚨 1020428569 507526415 2.01
tpch_q06/datafusion:vortex-file-compressed 🚨 545343455 335681613 1.62
tpch_q07/datafusion:vortex-file-compressed 🚨 779968396 494496354 1.58
tpch_q08/datafusion:vortex-file-compressed 🚨 931784600 707636270 1.32
tpch_q09/datafusion:vortex-file-compressed 551199998 496931712 1.11
tpch_q10/datafusion:vortex-file-compressed 🚨 746607655 560178496 1.33
tpch_q11/datafusion:vortex-file-compressed 🚨 454859729 331925587 1.37
tpch_q12/datafusion:vortex-file-compressed 🚨 669199246 386274679 1.73
tpch_q13/datafusion:vortex-file-compressed 🚨 329843217 219280353 1.50
tpch_q14/datafusion:vortex-file-compressed 🚨 468087428 254633768 1.84
tpch_q15/datafusion:vortex-file-compressed 🚨 721486221 484770919 1.49
tpch_q16/datafusion:vortex-file-compressed 🚨 411379231 258860285 1.59
tpch_q17/datafusion:vortex-file-compressed 🚨 651054649 400477879 1.63
tpch_q18/datafusion:vortex-file-compressed 408825249 325634927 1.26
tpch_q19/datafusion:vortex-file-compressed 724443128 581177928 1.25
tpch_q20/datafusion:vortex-file-compressed 647122949 557109828 1.16
tpch_q21/datafusion:vortex-file-compressed 876895948 752813018 1.16
tpch_q22/datafusion:vortex-file-compressed 237533184 227247020 1.05
datafusion / vortex-compact (1.204x ➖, 0↑ 13↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-compact 348946744 394850547 0.88
tpch_q02/datafusion:vortex-compact 521416011 657276106 0.79
tpch_q03/datafusion:vortex-compact 432315551 498280920 0.87
tpch_q04/datafusion:vortex-compact 🚨 332627884 226972184 1.47
tpch_q05/datafusion:vortex-compact 🚨 696908217 447802918 1.56
tpch_q06/datafusion:vortex-compact 🚨 471578513 339057311 1.39
tpch_q07/datafusion:vortex-compact 725644484 613637192 1.18
tpch_q08/datafusion:vortex-compact 🚨 1022118632 759110147 1.35
tpch_q09/datafusion:vortex-compact 791854034 768627552 1.03
tpch_q10/datafusion:vortex-compact 🚨 769532917 591075661 1.30
tpch_q11/datafusion:vortex-compact 🚨 579424913 421363770 1.38
tpch_q12/datafusion:vortex-compact 493151365 521628114 0.95
tpch_q13/datafusion:vortex-compact 🚨 272339836 193873479 1.40
tpch_q14/datafusion:vortex-compact 🚨 436372212 292273166 1.49
tpch_q15/datafusion:vortex-compact 🚨 779608105 497294331 1.57
tpch_q16/datafusion:vortex-compact 🚨 364711001 243839013 1.50
tpch_q17/datafusion:vortex-compact 🚨 736690908 411580194 1.79
tpch_q18/datafusion:vortex-compact 🚨 498453007 367493870 1.36
tpch_q19/datafusion:vortex-compact 🚨 1083170187 811818013 1.33
tpch_q20/datafusion:vortex-compact 515370888 626768917 0.82
tpch_q21/datafusion:vortex-compact 659288514 750282311 0.88
tpch_q22/datafusion:vortex-compact 222978355 232283392 0.96
datafusion / parquet (0.981x ➖, 2↑ 2↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/datafusion:parquet 286889617 231952395 1.24
tpch_q02/datafusion:parquet 472116351 443582157 1.06
tpch_q03/datafusion:parquet 391571563 348697043 1.12
tpch_q04/datafusion:parquet 194541243 196375862 0.99
tpch_q05/datafusion:parquet 463876850 446882997 1.04
tpch_q06/datafusion:parquet 136102583 152305138 0.89
tpch_q07/datafusion:parquet 470837209 512674473 0.92
tpch_q08/datafusion:parquet 530186351 572220120 0.93
tpch_q09/datafusion:parquet 465080095 544194788 0.85
tpch_q10/datafusion:parquet 526631431 650967223 0.81
tpch_q11/datafusion:parquet 🚀 401881403 611038198 0.66
tpch_q12/datafusion:parquet 271672924 283425999 0.96
tpch_q13/datafusion:parquet 431812466 525167108 0.82
tpch_q14/datafusion:parquet 283982050 269844896 1.05
tpch_q15/datafusion:parquet 440717864 447666949 0.98
tpch_q16/datafusion:parquet 280674048 270751101 1.04
tpch_q17/datafusion:parquet 505080980 541243450 0.93
tpch_q18/datafusion:parquet 🚨 637202520 484042264 1.32
tpch_q19/datafusion:parquet 520068687 407590337 1.28
tpch_q20/datafusion:parquet 🚀 461454349 790364802 0.58
tpch_q21/datafusion:parquet 631090506 539578918 1.17
tpch_q22/datafusion:parquet 🚨 242926118 176255335 1.38
duckdb / vortex-file-compressed (1.034x ➖, 0↑ 4↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-file-compressed 🚨 472285583 350060957 1.35
tpch_q02/duckdb:vortex-file-compressed 1162226610 1266202585 0.92
tpch_q03/duckdb:vortex-file-compressed 768731991 808440307 0.95
tpch_q04/duckdb:vortex-file-compressed 571422929 524276074 1.09
tpch_q05/duckdb:vortex-file-compressed 1112223113 1055350164 1.05
tpch_q06/duckdb:vortex-file-compressed 🚨 544658443 403025305 1.35
tpch_q07/duckdb:vortex-file-compressed 1208872248 1358961372 0.89
tpch_q08/duckdb:vortex-file-compressed 1261674149 1330489943 0.95
tpch_q09/duckdb:vortex-file-compressed 1172328563 1129908773 1.04
tpch_q10/duckdb:vortex-file-compressed 884005918 983466821 0.90
tpch_q11/duckdb:vortex-file-compressed 637572283 694978667 0.92
tpch_q12/duckdb:vortex-file-compressed 880500766 874625094 1.01
tpch_q13/duckdb:vortex-file-compressed 570024976 484573221 1.18
tpch_q14/duckdb:vortex-file-compressed 473527980 600081230 0.79
tpch_q15/duckdb:vortex-file-compressed 354739594 397278535 0.89
tpch_q16/duckdb:vortex-file-compressed 371882467 422474240 0.88
tpch_q17/duckdb:vortex-file-compressed 865600200 911191964 0.95
tpch_q18/duckdb:vortex-file-compressed 🚨 817410815 615674853 1.33
tpch_q19/duckdb:vortex-file-compressed 🚨 718275628 533139819 1.35
tpch_q20/duckdb:vortex-file-compressed 1007191978 1054956194 0.95
tpch_q21/duckdb:vortex-file-compressed 1493734409 1267028399 1.18
tpch_q22/duckdb:vortex-file-compressed 368240904 327107134 1.13
duckdb / vortex-compact (0.967x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-compact 346031605 414237341 0.84
tpch_q02/duckdb:vortex-compact 1138952226 1136519066 1.00
tpch_q03/duckdb:vortex-compact 833130406 650045721 1.28
tpch_q04/duckdb:vortex-compact 479676545 487778480 0.98
tpch_q05/duckdb:vortex-compact 1028224463 1053492842 0.98
tpch_q06/duckdb:vortex-compact 527157951 528288729 1.00
tpch_q07/duckdb:vortex-compact 1076372876 1216415579 0.88
tpch_q08/duckdb:vortex-compact 1242760849 1241480247 1.00
tpch_q09/duckdb:vortex-compact 1103193803 1182430778 0.93
tpch_q10/duckdb:vortex-compact 863819102 835367141 1.03
tpch_q11/duckdb:vortex-compact 628084570 668530840 0.94
tpch_q12/duckdb:vortex-compact 667338632 837864936 0.80
tpch_q13/duckdb:vortex-compact 366041686 515915485 0.71
tpch_q14/duckdb:vortex-compact 502353900 493490088 1.02
tpch_q15/duckdb:vortex-compact 368968843 366003922 1.01
tpch_q16/duckdb:vortex-compact 402110024 386799250 1.04
tpch_q17/duckdb:vortex-compact 870900685 775542696 1.12
tpch_q18/duckdb:vortex-compact 595543047 603730087 0.99
tpch_q19/duckdb:vortex-compact 669002962 678741984 0.99
tpch_q20/duckdb:vortex-compact 973138716 982567459 0.99
tpch_q21/duckdb:vortex-compact 1224545513 1175507347 1.04
tpch_q22/duckdb:vortex-compact 309126760 359137088 0.86
duckdb / parquet (1.043x ➖, 0↑ 1↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/duckdb:parquet 🚨 737613375 536588093 1.37
tpch_q02/duckdb:parquet 1267471601 1192273382 1.06
tpch_q03/duckdb:parquet 1347182904 1317721919 1.02
tpch_q04/duckdb:parquet 783568608 752057225 1.04
tpch_q05/duckdb:parquet 1648671252 1503983637 1.10
tpch_q06/duckdb:parquet 581163507 611219070 0.95
tpch_q07/duckdb:parquet 1528228384 1454903270 1.05
tpch_q08/duckdb:parquet 1879231801 1574359265 1.19
tpch_q09/duckdb:parquet 1584921640 1485076447 1.07
tpch_q10/duckdb:parquet 1660935870 1695716413 0.98
tpch_q11/duckdb:parquet 805647155 851220368 0.95
tpch_q12/duckdb:parquet 942248064 885368744 1.06
tpch_q13/duckdb:parquet 1032629846 910326315 1.13
tpch_q14/duckdb:parquet 986560134 896116633 1.10
tpch_q15/duckdb:parquet 653848530 731373820 0.89
tpch_q16/duckdb:parquet 682067330 826064476 0.83
tpch_q17/duckdb:parquet 1015487444 926117595 1.10
tpch_q18/duckdb:parquet 1153928927 986024065 1.17
tpch_q19/duckdb:parquet 926998234 1019930685 0.91
tpch_q20/duckdb:parquet 1311455777 1337369495 0.98
tpch_q21/duckdb:parquet 1434913300 1359428976 1.06
tpch_q22/duckdb:parquet 687255132 648776198 1.06

@AdamGS AdamGS marked this pull request as ready for review June 10, 2026 11:53
@AdamGS AdamGS requested review from a team and joseph-isaacs June 10, 2026 11:53
@joseph-isaacs joseph-isaacs merged commit 0a41704 into develop Jun 10, 2026
85 of 86 checks passed
@joseph-isaacs joseph-isaacs deleted the adamg/benchmarks-baseline branch June 10, 2026 12:05
@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: Random Access

Vortex (geomean): 0.909x ➖
Parquet (geomean): 0.983x ➖

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

unknown / unknown (0.990x ➖, 7↑ 1↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
random-access/feature-vectors/correlated/lance-tokio-local-disk 🚨 1562111 377760 4.14
random-access/feature-vectors/correlated/lance-tokio-local-disk-footer 1063292 1045440 1.02
random-access/feature-vectors/correlated/parquet-tokio-local-disk 8255472154 8393037554 0.98
random-access/feature-vectors/correlated/parquet-tokio-local-disk-footer 8185401678 8256868248 0.99
random-access/feature-vectors/correlated/vortex-tokio-local-disk 3720669 3677441 1.01
random-access/feature-vectors/correlated/vortex-tokio-local-disk-footer 🚀 2585009 3003978 0.86
random-access/feature-vectors/uniform/lance-tokio-local-disk 1190921 1167613 1.02
random-access/feature-vectors/uniform/lance-tokio-local-disk-footer 1846393 1833543 1.01
random-access/feature-vectors/uniform/parquet-tokio-local-disk 8210676408 8283933953 0.99
random-access/feature-vectors/uniform/parquet-tokio-local-disk-footer 8292282311 8263844454 1.00
random-access/feature-vectors/uniform/vortex-tokio-local-disk 3452166 3562723 0.97
random-access/feature-vectors/uniform/vortex-tokio-local-disk-footer 5820804 6041774 0.96
random-access/lance-tokio-local-disk 639928 629709 1.02
random-access/lance-tokio-local-disk-footer 1298113 1286369 1.01
random-access/nested-lists/correlated/lance-tokio-local-disk 236228 241915 0.98
random-access/nested-lists/correlated/lance-tokio-local-disk-footer 583430 584548 1.00
random-access/nested-lists/correlated/parquet-tokio-local-disk 130737270 129588478 1.01
random-access/nested-lists/correlated/parquet-tokio-local-disk-footer 129896254 128957902 1.01
random-access/nested-lists/correlated/vortex-tokio-local-disk 462157 489448 0.94
random-access/nested-lists/correlated/vortex-tokio-local-disk-footer 🚀 559908 641030 0.87
random-access/nested-lists/uniform/lance-tokio-local-disk 1001081 1002743 1.00
random-access/nested-lists/uniform/lance-tokio-local-disk-footer 1351706 1364325 0.99
random-access/nested-lists/uniform/parquet-tokio-local-disk 130652820 130395072 1.00
random-access/nested-lists/uniform/parquet-tokio-local-disk-footer 129772472 129748020 1.00
random-access/nested-lists/uniform/vortex-tokio-local-disk 1916046 1992673 0.96
random-access/nested-lists/uniform/vortex-tokio-local-disk-footer 2025947 2108381 0.96
random-access/nested-structs/correlated/lance-tokio-local-disk 365585 366367 1.00
random-access/nested-structs/correlated/lance-tokio-local-disk-footer 546251 557082 0.98
random-access/nested-structs/correlated/parquet-tokio-local-disk 22376159 23346683 0.96
random-access/nested-structs/correlated/parquet-tokio-local-disk-footer 22733347 23329111 0.97
random-access/nested-structs/correlated/vortex-tokio-local-disk 🚀 534957 595731 0.90
random-access/nested-structs/correlated/vortex-tokio-local-disk-footer 701845 770392 0.91
random-access/nested-structs/uniform/lance-tokio-local-disk 2628591 2497128 1.05
random-access/nested-structs/uniform/lance-tokio-local-disk-footer 2758444 2678601 1.03
random-access/nested-structs/uniform/parquet-tokio-local-disk 22743329 23400086 0.97
random-access/nested-structs/uniform/parquet-tokio-local-disk-footer 22510812 23050585 0.98
random-access/nested-structs/uniform/vortex-tokio-local-disk 1324744 1417100 0.93
random-access/nested-structs/uniform/vortex-tokio-local-disk-footer 1556505 1657194 0.94
random-access/parquet-tokio-local-disk 165336885 170150063 0.97
random-access/parquet-tokio-local-disk-footer 165169333 169569666 0.97
random-access/taxi/correlated/lance-tokio-local-disk 937545 937184 1.00
random-access/taxi/correlated/lance-tokio-local-disk-footer 1815178 1848336 0.98
random-access/taxi/correlated/parquet-tokio-local-disk 247707851 254863289 0.97
random-access/taxi/correlated/parquet-tokio-local-disk-footer 248467227 255401658 0.97
random-access/taxi/correlated/vortex-tokio-local-disk 🚀 1022266 1282884 0.80
random-access/taxi/correlated/vortex-tokio-local-disk-footer 🚀 1397769 1800329 0.78
random-access/taxi/uniform/lance-tokio-local-disk 9354776 9227596 1.01
random-access/taxi/uniform/lance-tokio-local-disk-footer 9891122 9885689 1.00
random-access/taxi/uniform/parquet-tokio-local-disk 263052890 271010298 0.97
random-access/taxi/uniform/parquet-tokio-local-disk-footer 263298542 271749126 0.97
random-access/taxi/uniform/vortex-tokio-local-disk 3389270 3663090 0.93
random-access/taxi/uniform/vortex-tokio-local-disk-footer 3881754 4238931 0.92
random-access/vortex-tokio-local-disk 🚀 667787 758720 0.88
random-access/vortex-tokio-local-disk-footer 🚀 1033916 1185781 0.87

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: Appian on NVME

Verdict: No clear signal (low confidence)
Attributed Vortex impact: -1.7%
Engines: DataFusion No clear signal (-3.8%, environment too noisy confidence) · DuckDB No clear signal (-0.3%, low confidence)
Vortex (geomean): 0.898x ✅
Parquet (geomean): 0.919x ➖
Shifts: Parquet (control) -8.1% · Median polish -9.5%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (0.890x ✅, 5↑ 0↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
appian_q01/datafusion:vortex-file-compressed 125882879 137057916 0.92
appian_q02/datafusion:vortex-file-compressed 🚀 582206623 673558534 0.86
appian_q03/datafusion:vortex-file-compressed 🚀 332962275 390677631 0.85
appian_q04/datafusion:vortex-file-compressed 44229567442 48372933568 0.91
appian_q05/datafusion:vortex-file-compressed 🚀 248597571 281422674 0.88
appian_q06/datafusion:vortex-file-compressed 327149999 360610588 0.91
appian_q07/datafusion:vortex-file-compressed 🚀 409802308 462177772 0.89
appian_q08/datafusion:vortex-file-compressed 🚀 1937130543 2155005722 0.90
datafusion / parquet (0.925x ➖, 4↑ 0↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
appian_q01/datafusion:parquet 152335273 141580778 1.08
appian_q02/datafusion:parquet 644074167 668780491 0.96
appian_q03/datafusion:parquet 361884092 368950535 0.98
appian_q04/datafusion:parquet 44298334840 48295878957 0.92
appian_q05/datafusion:parquet 🚀 272119816 309349726 0.88
appian_q06/datafusion:parquet 🚀 349236887 397460137 0.88
appian_q07/datafusion:parquet 🚀 431923249 507507428 0.85
appian_q08/datafusion:parquet 🚀 1916687891 2185678932 0.88
duckdb / vortex-file-compressed (0.906x ➖, 5↑ 0↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
appian_q01/duckdb:vortex-file-compressed 🚀 192525650 214384258 0.90
appian_q02/duckdb:vortex-file-compressed 🚀 605336720 684310169 0.88
appian_q03/duckdb:vortex-file-compressed 258210819 286228955 0.90
appian_q04/duckdb:vortex-file-compressed 🚀 1307818900 1459969651 0.90
appian_q05/duckdb:vortex-file-compressed 🚀 289577264 328389188 0.88
appian_q06/duckdb:vortex-file-compressed 800033062 815977872 0.98
appian_q07/duckdb:vortex-file-compressed 339794058 370444848 0.92
appian_q08/duckdb:vortex-file-compressed 🚀 1279578285 1428707732 0.90
duckdb / parquet (0.912x ➖, 3↑ 0↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
appian_q01/duckdb:parquet 202257804 220773831 0.92
appian_q02/duckdb:parquet 🚀 600913153 672408052 0.89
appian_q03/duckdb:parquet 289319340 315202718 0.92
appian_q04/duckdb:parquet 1319687468 1452546280 0.91
appian_q05/duckdb:parquet 🚀 312315661 348526933 0.90
appian_q06/duckdb:parquet 788099226 812010230 0.97
appian_q07/duckdb:parquet 362342968 400320172 0.91
appian_q08/duckdb:parquet 🚀 1258066569 1411487308 0.89
duckdb / duckdb (0.912x ➖, 3↑ 0↓)
name PR 6a4663d (ns) base 47d2041 (ns) ratio (PR/base)
appian_q01/duckdb:duckdb 🚀 170252101 190965161 0.89
appian_q02/duckdb:duckdb 549384598 608295891 0.90
appian_q03/duckdb:duckdb 385387885 408596882 0.94
appian_q04/duckdb:duckdb 1300084557 1440759902 0.90
appian_q05/duckdb:duckdb 🚀 275302922 314957173 0.87
appian_q06/duckdb:duckdb 781073438 806333490 0.97
appian_q07/duckdb:duckdb 325506602 353773478 0.92
appian_q08/duckdb:duckdb 🚀 1223320662 1359661418 0.90

File Size Changes (4 files changed, -0.0% overall, 1↑ 3↓)
File Scale Format Base HEAD Change %
orderview.vortex 1.0 vortex-file-compressed 81.11 MB 81.14 MB +26.38 KB +0.0%
addressview.vortex 1.0 vortex-file-compressed 34.62 MB 34.62 MB 24 B -0.0%
creditcardview.vortex 1.0 vortex-file-compressed 60.99 MB 60.98 MB 16.11 KB -0.0%
duckdb.db 1.0 vortex-compact 268.00 KB 0 B 268.00 KB -100.0%

Totals:

  • vortex-compact: 271.96 MB → 271.69 MB (-0.1%)
  • vortex-file-compressed: 513.73 MB → 513.74 MB (+0.0%)

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: TPC-H SF=10 on S3

Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -7.2%
Engines: DataFusion No clear signal (-12.8%, environment too noisy confidence) · DuckDB No clear signal (-1.3%, environment too noisy confidence)
Vortex (geomean): 0.983x ➖
Parquet (geomean): 1.059x ➖
Shifts: Parquet (control) +5.9% · Median polish -0.6%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.049x ➖, 0↑ 1↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-file-compressed 891060958 790317687 1.13
tpch_q02/datafusion:vortex-file-compressed 1410458654 1226436383 1.15
tpch_q03/datafusion:vortex-file-compressed 1179167243 1063813387 1.11
tpch_q04/datafusion:vortex-file-compressed 649153390 593146925 1.09
tpch_q05/datafusion:vortex-file-compressed 1198917009 985825257 1.22
tpch_q06/datafusion:vortex-file-compressed 672715711 603639942 1.11
tpch_q07/datafusion:vortex-file-compressed 1312577069 1253241308 1.05
tpch_q08/datafusion:vortex-file-compressed 1607000487 1679637099 0.96
tpch_q09/datafusion:vortex-file-compressed 1499113382 1408196001 1.06
tpch_q10/datafusion:vortex-file-compressed 1439530010 1378458768 1.04
tpch_q11/datafusion:vortex-file-compressed 🚨 809425048 592262448 1.37
tpch_q12/datafusion:vortex-file-compressed 1089577073 1213574093 0.90
tpch_q13/datafusion:vortex-file-compressed 537922115 622883561 0.86
tpch_q14/datafusion:vortex-file-compressed 643469956 684394740 0.94
tpch_q15/datafusion:vortex-file-compressed 1142338020 1355029689 0.84
tpch_q16/datafusion:vortex-file-compressed 709452046 642373304 1.10
tpch_q17/datafusion:vortex-file-compressed 1535211536 1608866084 0.95
tpch_q18/datafusion:vortex-file-compressed 1498118205 1701036813 0.88
tpch_q19/datafusion:vortex-file-compressed 1574825311 1325633467 1.19
tpch_q20/datafusion:vortex-file-compressed 1397438031 1311297405 1.07
tpch_q21/datafusion:vortex-file-compressed 1882659365 1805117141 1.04
tpch_q22/datafusion:vortex-file-compressed 907569502 771138151 1.18
datafusion / vortex-compact (0.936x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-compact 852368301 859129884 0.99
tpch_q02/datafusion:vortex-compact 1112616868 1470373883 0.76
tpch_q03/datafusion:vortex-compact 1141331669 1250961803 0.91
tpch_q04/datafusion:vortex-compact 672250914 691551586 0.97
tpch_q05/datafusion:vortex-compact 1217571267 1205783954 1.01
tpch_q06/datafusion:vortex-compact 690423706 747513070 0.92
tpch_q07/datafusion:vortex-compact 1066475249 1087119252 0.98
tpch_q08/datafusion:vortex-compact 1523693532 1596717986 0.95
tpch_q09/datafusion:vortex-compact 1427517692 1520597461 0.94
tpch_q10/datafusion:vortex-compact 1251243448 1258865956 0.99
tpch_q11/datafusion:vortex-compact 613649160 790718942 0.78
tpch_q12/datafusion:vortex-compact 1001562250 973664203 1.03
tpch_q13/datafusion:vortex-compact 540021957 554942398 0.97
tpch_q14/datafusion:vortex-compact 702819084 626155764 1.12
tpch_q15/datafusion:vortex-compact 1292645919 1305873561 0.99
tpch_q16/datafusion:vortex-compact 595358560 573898025 1.04
tpch_q17/datafusion:vortex-compact 1130930956 1464407896 0.77
tpch_q18/datafusion:vortex-compact 1129116499 1203090530 0.94
tpch_q19/datafusion:vortex-compact 1179025541 1215260743 0.97
tpch_q20/datafusion:vortex-compact 1281821162 1322894870 0.97
tpch_q21/datafusion:vortex-compact 1515941992 1746760976 0.87
tpch_q22/datafusion:vortex-compact 850833952 1053661973 0.81
datafusion / parquet (1.137x ➖, 0↑ 6↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/datafusion:parquet 915973157 934743222 0.98
tpch_q02/datafusion:parquet 1078049465 954975574 1.13
tpch_q03/datafusion:parquet 🚨 1239634468 930890480 1.33
tpch_q04/datafusion:parquet 626229355 531362448 1.18
tpch_q05/datafusion:parquet 1441227152 1208608921 1.19
tpch_q06/datafusion:parquet 588032780 531636167 1.11
tpch_q07/datafusion:parquet 1530405311 1520392720 1.01
tpch_q08/datafusion:parquet 1916032853 1982640558 0.97
tpch_q09/datafusion:parquet 🚨 2529638896 1918300485 1.32
tpch_q10/datafusion:parquet 3034044540 2695565191 1.13
tpch_q11/datafusion:parquet 🚨 1346682892 738363892 1.82
tpch_q12/datafusion:parquet 🚨 1099249817 713985581 1.54
tpch_q13/datafusion:parquet 🚨 1109219337 751502797 1.48
tpch_q14/datafusion:parquet 🚨 1116169822 820135567 1.36
tpch_q15/datafusion:parquet 1852382866 1513488328 1.22
tpch_q16/datafusion:parquet 741121769 660256288 1.12
tpch_q17/datafusion:parquet 1626906665 1710120734 0.95
tpch_q18/datafusion:parquet 2076451846 1877316831 1.11
tpch_q19/datafusion:parquet 1158738320 1193115661 0.97
tpch_q20/datafusion:parquet 2060963735 2535781204 0.81
tpch_q21/datafusion:parquet 2178043080 2354211966 0.93
tpch_q22/datafusion:parquet 1426172593 1678117929 0.85
duckdb / vortex-file-compressed (0.983x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-file-compressed 726886884 662523689 1.10
tpch_q02/duckdb:vortex-file-compressed 1448737017 1597728948 0.91
tpch_q03/duckdb:vortex-file-compressed 1253342516 1159846525 1.08
tpch_q04/duckdb:vortex-file-compressed 985000578 991478323 0.99
tpch_q05/duckdb:vortex-file-compressed 1468731179 1429370584 1.03
tpch_q06/duckdb:vortex-file-compressed 665331831 701330709 0.95
tpch_q07/duckdb:vortex-file-compressed 1513366018 1544552837 0.98
tpch_q08/duckdb:vortex-file-compressed 1814843247 1905405486 0.95
tpch_q09/duckdb:vortex-file-compressed 2171826013 2074796838 1.05
tpch_q10/duckdb:vortex-file-compressed 1477544035 1767062318 0.84
tpch_q11/duckdb:vortex-file-compressed 936431931 777221627 1.20
tpch_q12/duckdb:vortex-file-compressed 1421121782 1536450342 0.92
tpch_q13/duckdb:vortex-file-compressed 914852468 1118677324 0.82
tpch_q14/duckdb:vortex-file-compressed 809446244 789891012 1.02
tpch_q15/duckdb:vortex-file-compressed 678918716 644174510 1.05
tpch_q16/duckdb:vortex-file-compressed 613912334 736129051 0.83
tpch_q17/duckdb:vortex-file-compressed 1115759861 1255982797 0.89
tpch_q18/duckdb:vortex-file-compressed 1285990319 1067523411 1.20
tpch_q19/duckdb:vortex-file-compressed 870681042 1034422389 0.84
tpch_q20/duckdb:vortex-file-compressed 1604770168 1618925129 0.99
tpch_q21/duckdb:vortex-file-compressed 2296385352 2417063768 0.95
tpch_q22/duckdb:vortex-file-compressed 646401771 563227749 1.15
duckdb / vortex-compact (0.966x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-compact 729570789 674515248 1.08
tpch_q02/duckdb:vortex-compact 1264122375 1399370962 0.90
tpch_q03/duckdb:vortex-compact 922558658 1116017754 0.83
tpch_q04/duckdb:vortex-compact 679975100 769555127 0.88
tpch_q05/duckdb:vortex-compact 1352101675 1301047187 1.04
tpch_q06/duckdb:vortex-compact 651160906 589898571 1.10
tpch_q07/duckdb:vortex-compact 1383031796 1307303151 1.06
tpch_q08/duckdb:vortex-compact 1609011225 1605497538 1.00
tpch_q09/duckdb:vortex-compact 1797892081 2074717700 0.87
tpch_q10/duckdb:vortex-compact 1322516764 1350694446 0.98
tpch_q11/duckdb:vortex-compact 714879143 819634332 0.87
tpch_q12/duckdb:vortex-compact 1222738153 1432766389 0.85
tpch_q13/duckdb:vortex-compact 845500714 1018385361 0.83
tpch_q14/duckdb:vortex-compact 749149938 772914618 0.97
tpch_q15/duckdb:vortex-compact 585751256 569252604 1.03
tpch_q16/duckdb:vortex-compact 488585577 471189319 1.04
tpch_q17/duckdb:vortex-compact 1050146993 1063840178 0.99
tpch_q18/duckdb:vortex-compact 934721290 944421533 0.99
tpch_q19/duckdb:vortex-compact 805569067 936035545 0.86
tpch_q20/duckdb:vortex-compact 1527906003 1352999881 1.13
tpch_q21/duckdb:vortex-compact 2041828756 2114533625 0.97
tpch_q22/duckdb:vortex-compact 609037948 557085637 1.09
duckdb / parquet (0.987x ➖, 0↑ 0↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
tpch_q01/duckdb:parquet 930115878 963637944 0.97
tpch_q02/duckdb:parquet 1790812081 1763397504 1.02
tpch_q03/duckdb:parquet 2597701843 2088674263 1.24
tpch_q04/duckdb:parquet 1527664431 1402629798 1.09
tpch_q05/duckdb:parquet 2616577528 2451780290 1.07
tpch_q06/duckdb:parquet 1356067589 1245570321 1.09
tpch_q07/duckdb:parquet 2177050520 2332539278 0.93
tpch_q08/duckdb:parquet 3498451691 2942393428 1.19
tpch_q09/duckdb:parquet 3748827799 3617084461 1.04
tpch_q10/duckdb:parquet 4147577912 4011154711 1.03
tpch_q11/duckdb:parquet 1233507342 1229990374 1.00
tpch_q12/duckdb:parquet 1574796640 1841087485 0.86
tpch_q13/duckdb:parquet 1545305317 1976345495 0.78
tpch_q14/duckdb:parquet 1679518422 1769458562 0.95
tpch_q15/duckdb:parquet 1245646185 1325191334 0.94
tpch_q16/duckdb:parquet 1104444813 1162695410 0.95
tpch_q17/duckdb:parquet 1552230877 1873192718 0.83
tpch_q18/duckdb:parquet 2059407662 2128302957 0.97
tpch_q19/duckdb:parquet 1955739594 2045267028 0.96
tpch_q20/duckdb:parquet 2873483425 2636343280 1.09
tpch_q21/duckdb:parquet 2480879841 2386105306 1.04
tpch_q22/duckdb:parquet 1345125338 1613042882 0.83

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: Compression

Vortex (geomean): 0.997x ➖
Parquet (geomean): 0.984x ➖

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

unknown / unknown (0.979x ➖, 4↑ 2↓)
name PR 6a4663d (ns) base 09cd0e2 (ns) ratio (PR/base)
compress time/Arade 1091135619 1168546240 0.93
compress time/Bimbo 5536426371 5938521567 0.93
compress time/CMSprovider 2750228223 2859603851 0.96
compress time/Euro2016 469019290 491851058 0.95
compress time/Food 380074966 392950512 0.97
compress time/HashTags 891057196 866699100 1.03
compress time/TPC-H l_comment canonical 1221076491 1279661563 0.95
compress time/TPC-H l_comment chunked 1234330587 1280688910 0.96
compress time/taxi 661421066 719644929 0.92
compress time/wide table cols=100 chunks=1 rows=1000 11623720 12551135 0.93
compress time/wide table cols=100 chunks=50 rows=1000 11813934 12967486 0.91
compress time/wide table cols=1000 chunks=1 rows=1000 🚀 122051034 136200598 0.90
compress time/wide table cols=1000 chunks=50 rows=1000 124082269 137549891 0.90
compress time/wide table cols=10000 chunks=1 rows=1000 1472181507 1491176033 0.99
compress time/wide table cols=10000 chunks=50 rows=1000 1482214069 1517886123 0.98
compress time/wide table cols=100000 chunks=10 rows=1000 project=10000 15329726778 nan
decompress time/Arade 27637674 28119596 0.98
decompress time/Bimbo 77435004 83392219 0.93
decompress time/CMSprovider 84191378 79365952 1.06
decompress time/Euro2016 19981454 21206458 0.94
decompress time/Food 8748693 9085912 0.96
decompress time/HashTags 89673748 91165846 0.98
decompress time/TPC-H l_comment canonical 38004660 40569785 0.94
decompress time/TPC-H l_comment chunked 39789064 41109192 0.97
decompress time/taxi 14368260 15651646 0.92
decompress time/wide table cols=100 chunks=1 rows=1000 2450066 2457123 1.00
decompress time/wide table cols=100 chunks=50 rows=1000 2463718 2666256 0.92
decompress time/wide table cols=1000 chunks=1 rows=1000 21458409 22654347 0.95
decompress time/wide table cols=1000 chunks=50 rows=1000 22001978 22804927 0.96
decompress time/wide table cols=10000 chunks=1 rows=1000 235337072 247748983 0.95
decompress time/wide table cols=10000 chunks=50 rows=1000 227980152 246978569 0.92
decompress time/wide table cols=100000 chunks=10 rows=1000 project=10000 6173244588 nan
parquet size/Arade 258014282 258014282 1.00
parquet size/Bimbo 384517292 384517292 1.00
parquet size/CMSprovider 376885545 376885545 1.00
parquet size/Euro2016 122975499 122975499 1.00
parquet size/Food 35699500 35699500 1.00
parquet size/HashTags 133510943 133510943 1.00
parquet size/TPC-H l_comment canonical 158358238 158358238 1.00
parquet size/TPC-H l_comment chunked 158358238 158358238 1.00
parquet size/taxi 55283635 55283635 1.00
parquet size/wide table cols=100 chunks=1 rows=1000 932404 932404 1.00
parquet size/wide table cols=100 chunks=50 rows=1000 932404 932404 1.00
parquet size/wide table cols=1000 chunks=1 rows=1000 9324004 9324004 1.00
parquet size/wide table cols=1000 chunks=50 rows=1000 9324004 9324004 1.00
parquet size/wide table cols=10000 chunks=1 rows=1000 93240004 93240004 1.00
parquet size/wide table cols=10000 chunks=50 rows=1000 93240004 93240004 1.00
parquet size/wide table cols=100000 chunks=10 rows=1000 project=10000 932400004 nan
parquet_rs-zstd compress time/Arade 2615080332 2702948453 0.97
parquet_rs-zstd compress time/Bimbo 12720500147 12933617299 0.98
parquet_rs-zstd compress time/CMSprovider 6858420241 7441182221 0.92
parquet_rs-zstd compress time/Euro2016 1356971841 1363693278 1.00
parquet_rs-zstd compress time/Food 835127368 818555332 1.02
parquet_rs-zstd compress time/HashTags 2287059405 2380652383 0.96
parquet_rs-zstd compress time/TPC-H l_comment canonical 3236631242 3362234089 0.96
parquet_rs-zstd compress time/TPC-H l_comment chunked 3229508626 3332206133 0.97
parquet_rs-zstd compress time/taxi 1196598297 1225334477 0.98
parquet_rs-zstd compress time/wide table cols=100 chunks=1 rows=1000 🚀 6205850 7170677 0.87
parquet_rs-zstd compress time/wide table cols=100 chunks=50 rows=1000 🚀 6225155 7219332 0.86
parquet_rs-zstd compress time/wide table cols=1000 chunks=1 rows=1000 75375837 77442812 0.97
parquet_rs-zstd compress time/wide table cols=1000 chunks=50 rows=1000 75494261 79432428 0.95
parquet_rs-zstd compress time/wide table cols=10000 chunks=1 rows=1000 775664966 811610035 0.96
parquet_rs-zstd compress time/wide table cols=10000 chunks=50 rows=1000 780264516 816533866 0.96
parquet_rs-zstd compress time/wide table cols=100000 chunks=10 rows=1000 project=10000 9475944299 nan
parquet_rs-zstd decompress time/Arade 616556947 639916674 0.96
parquet_rs-zstd decompress time/Bimbo 1690876945 1723690152 0.98
parquet_rs-zstd decompress time/CMSprovider 1724455992 1792885931 0.96
parquet_rs-zstd decompress time/Euro2016 397031832 387936095 1.02
parquet_rs-zstd decompress time/Food 203795859 203865278 1.00
parquet_rs-zstd decompress time/HashTags 🚀 656196711 735058707 0.89
parquet_rs-zstd decompress time/TPC-H l_comment canonical 586130407 615300602 0.95
parquet_rs-zstd decompress time/TPC-H l_comment chunked 588809281 622629601 0.95
parquet_rs-zstd decompress time/taxi 244268268 249443051 0.98
parquet_rs-zstd decompress time/wide table cols=100 chunks=1 rows=1000 2838895 3027511 0.94
parquet_rs-zstd decompress time/wide table cols=100 chunks=50 rows=1000 2844986 3067550 0.93
parquet_rs-zstd decompress time/wide table cols=1000 chunks=1 rows=1000 32937211 33626504 0.98
parquet_rs-zstd decompress time/wide table cols=1000 chunks=50 rows=1000 33321840 35972706 0.93
parquet_rs-zstd decompress time/wide table cols=10000 chunks=1 rows=1000 349921185 353979626 0.99
parquet_rs-zstd decompress time/wide table cols=10000 chunks=50 rows=1000 357338555 354520115 1.01
parquet_rs-zstd decompress time/wide table cols=100000 chunks=10 rows=1000 project=10000 605125896 nan
vortex-file-compressed size/Arade 145363860 145363860 1.00
vortex-file-compressed size/Bimbo 468763396 468763396 1.00
vortex-file-compressed size/CMSprovider 417904836 417904812 1.00
vortex-file-compressed size/Euro2016 158523740 158720228 1.00
vortex-file-compressed size/Food 41927000 41927000 1.00
vortex-file-compressed size/HashTags 185307492 185572572 1.00
vortex-file-compressed size/TPC-H l_comment canonical 178472712 178363800 1.00
vortex-file-compressed size/TPC-H l_comment chunked 178441104 178532224 1.00
vortex-file-compressed size/taxi 52364012 52364012 1.00
vortex-file-compressed size/wide table cols=100 chunks=1 rows=1000 930912 930912 1.00
vortex-file-compressed size/wide table cols=100 chunks=50 rows=1000 930912 930912 1.00
vortex-file-compressed size/wide table cols=1000 chunks=1 rows=1000 9293712 9293712 1.00
vortex-file-compressed size/wide table cols=1000 chunks=50 rows=1000 9293712 9293712 1.00
vortex-file-compressed size/wide table cols=10000 chunks=1 rows=1000 92957712 92957712 1.00
vortex-file-compressed size/wide table cols=10000 chunks=50 rows=1000 92957712 92957712 1.00
vortex-file-compressed size/wide table cols=100000 chunks=10 rows=1000 project=10000 929597712 nan
vortex:parquet-zstd ratio compress time/Arade 0 0 0.97
vortex:parquet-zstd ratio compress time/Bimbo 0 0 0.95
vortex:parquet-zstd ratio compress time/CMSprovider 0 0 1.04
vortex:parquet-zstd ratio compress time/Euro2016 0 0 0.96
vortex:parquet-zstd ratio compress time/Food 0 0 0.95
vortex:parquet-zstd ratio compress time/HashTags 0 0 1.07
vortex:parquet-zstd ratio compress time/TPC-H l_comment canonical 0 0 0.99
vortex:parquet-zstd ratio compress time/TPC-H l_comment chunked 0 0 0.99
vortex:parquet-zstd ratio compress time/taxi 0 0 0.94
vortex:parquet-zstd ratio compress time/wide table cols=100 chunks=1 rows=1000 1 1 1.07
vortex:parquet-zstd ratio compress time/wide table cols=100 chunks=50 rows=1000 1 1 1.06
vortex:parquet-zstd ratio compress time/wide table cols=1000 chunks=1 rows=1000 1 1 0.92
vortex:parquet-zstd ratio compress time/wide table cols=1000 chunks=50 rows=1000 1 1 0.95
vortex:parquet-zstd ratio compress time/wide table cols=10000 chunks=1 rows=1000 1 1 1.03
vortex:parquet-zstd ratio compress time/wide table cols=10000 chunks=50 rows=1000 1 1 1.02
vortex:parquet-zstd ratio compress time/wide table cols=100000 chunks=10 rows=1000 project=10000 1 nan
vortex:parquet-zstd ratio decompress time/Arade 0 0 1.02
vortex:parquet-zstd ratio decompress time/Bimbo 0 0 0.95
vortex:parquet-zstd ratio decompress time/CMSprovider 🚨 0 0 1.10
vortex:parquet-zstd ratio decompress time/Euro2016 0 0 0.92
vortex:parquet-zstd ratio decompress time/Food 0 0 0.96
vortex:parquet-zstd ratio decompress time/HashTags 🚨 0 0 1.10
vortex:parquet-zstd ratio decompress time/TPC-H l_comment canonical 0 0 0.98
vortex:parquet-zstd ratio decompress time/TPC-H l_comment chunked 0 0 1.02
vortex:parquet-zstd ratio decompress time/taxi 0 0 0.94
vortex:parquet-zstd ratio decompress time/wide table cols=100 chunks=1 rows=1000 0 0 1.06
vortex:parquet-zstd ratio decompress time/wide table cols=100 chunks=50 rows=1000 0 0 1.00
vortex:parquet-zstd ratio decompress time/wide table cols=1000 chunks=1 rows=1000 0 0 0.97
vortex:parquet-zstd ratio decompress time/wide table cols=1000 chunks=50 rows=1000 0 0 1.04
vortex:parquet-zstd ratio decompress time/wide table cols=10000 chunks=1 rows=1000 0 0 0.96
vortex:parquet-zstd ratio decompress time/wide table cols=10000 chunks=50 rows=1000 0 0 0.92
vortex:parquet-zstd ratio decompress time/wide table cols=100000 chunks=10 rows=1000 project=10000 10 nan
vortex:parquet-zstd size/Arade 0 0 1.00
vortex:parquet-zstd size/Bimbo 1 1 1.00
vortex:parquet-zstd size/CMSprovider 1 1 1.00
vortex:parquet-zstd size/Euro2016 1 1 1.00
vortex:parquet-zstd size/Food 1 1 1.00
vortex:parquet-zstd size/HashTags 1 1 1.00
vortex:parquet-zstd size/TPC-H l_comment canonical 1 1 1.00
vortex:parquet-zstd size/TPC-H l_comment chunked 1 1 1.00
vortex:parquet-zstd size/taxi 0 0 1.00
vortex:parquet-zstd size/wide table cols=100 chunks=1 rows=1000 0 0 1.00
vortex:parquet-zstd size/wide table cols=100 chunks=50 rows=1000 0 0 1.00
vortex:parquet-zstd size/wide table cols=1000 chunks=1 rows=1000 0 0 1.00
vortex:parquet-zstd size/wide table cols=1000 chunks=50 rows=1000 0 0 1.00
vortex:parquet-zstd size/wide table cols=10000 chunks=1 rows=1000 0 0 1.00
vortex:parquet-zstd size/wide table cols=10000 chunks=50 rows=1000 0 0 1.00
vortex:parquet-zstd size/wide table cols=100000 chunks=10 rows=1000 project=10000 0 nan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants