Repository navigation

perf: Optimize ExternalSorter with chunked sort pipeline and radix sort kernel - #21600

Closed

mbutrovich wants to merge 26 commits into

apache:mainfrom

mbutrovich:sort_redesign

Commits on Apr 9, 2026
- Bring over apache/arrow-rs/9683, integrate into sorts, add heuristic to choose between sort implementations.
  mbutrovich
  committed
  e1e1d09
Commits on Apr 13, 2026
- Merge branch 'main' into arrow_rs_9683
  mbutrovich
  committed
  9371dd1
- Stash with implementation, need to fix accounting for one test.
  mbutrovich
  committed
  fc35b08
- Fix more tests.
  mbutrovich
  committed
  d822a50
- Tests pass.
  mbutrovich
  committed
  54be475
- Cleanup.
  mbutrovich
  committed
  a42ba49
- More cleanup.
  mbutrovich
  committed
  24dbdb7
- More tests.
  mbutrovich
  committed
  e83ffc6
- More tests.
  mbutrovich
  committed
  60fbdfb
- Cleanup before pushing.
  mbutrovich
  committed
  8e8f774
- Fix CI failures.
  mbutrovich
  committed
  aa93a6e
- Fix configs.md.
  mbutrovich
  committed
  af514fe
- Fix CI failures.
  mbutrovich
  committed
  c527553
Commits on Apr 14, 2026
- Avoid radix sort for decimal types for now.
  mbutrovich
  committed
  83860c0
- Fix information_schema.slt test.
  mbutrovich
  committed
  a88eacf
- Update to latest radix kernel from arrow-rs PR.
  mbutrovich
  committed
  175dcf6
- Use lexsort for single columns, radix otherwise. Should help with Q11 regression.
  mbutrovich
  committed
  4bdf871
- Address some PR feedback.
  mbutrovich
  committed
  a1f0193
- Address some PR feedback.
  mbutrovich
  committed
  bbda50f
- Update failing test.
  mbutrovich
  committed
  2698b85
- Update failing test for realsies.
  mbutrovich
  committed
  d5d3ef7
- ## `datafusion/physical-plan/src/sorts/sort.rs`
  mbutrovich
  committed
  e0fb0a8
- Cleanup.
  mbutrovich
  committed
  73bb06b
- Fix copy.slt test to show new metrics.
  mbutrovich
  committed
  7a75c38
- Temporarily default radix to false to get benchmarks.
  mbutrovich
  committed
  482e72c
- Revert "Temporarily default radix to false to get benchmarks."
  mbutrovich
  committed
  96dd0df

Navigation Menu

Repository navigation

perf: Optimize ExternalSorter with chunked sort pipeline and radix sort kernel - #21600

perf: Optimize ExternalSorter with chunked sort pipeline and radix sort kernel#21600
mbutrovich wants to merge 26 commits into
apache:mainfrom
mbutrovich:sort_redesign

Commits on Apr 9, 2026

Bring over apache/arrow-rs/9683, integrate into sorts, add heuristic to choose between sort implementations.

Commits on Apr 13, 2026

Merge branch 'main' into arrow_rs_9683

Stash with implementation, need to fix accounting for one test.

Fix more tests.

Tests pass.

Cleanup.

More cleanup.

More tests.

More tests.

Cleanup before pushing.

Fix CI failures.

Fix configs.md.

Fix CI failures.

Commits on Apr 14, 2026

Avoid radix sort for decimal types for now.

Fix information_schema.slt test.

Update to latest radix kernel from arrow-rs PR.

Use lexsort for single columns, radix otherwise. Should help with Q11 regression.

Address some PR feedback.

Address some PR feedback.

Update failing test.

Update failing test for realsies.

## `datafusion/physical-plan/src/sorts/sort.rs`

Cleanup.

Fix copy.slt test to show new metrics.

Temporarily default radix to false to get benchmarks.

Revert "Temporarily default radix to false to get benchmarks."