Repository navigation
Release arrow-rs / parquet Major version 59.0.0 (May 2026) #9110
Description
Activity
- changed the title
[-]Release arrow-rs / parquet Major version 59.0.0 (April 2026)[/-][+]Release arrow-rs / parquet Major version `59.0.0` (April 2026)[/+]on Jan 7, 2026 - changed the title
[-]Release arrow-rs / parquet Major version `59.0.0` (April 2026)[/-][+]Release arrow-rs / parquet Major version `59.0.0` (May 2026)[/+]on Feb 23, 2026 Given that the 58.0.0 release didn't go out until almost the end of February, moving this date back by a month
- added a commit that references this issue
on May 13, 2026 Would love to see #9203 in the 59 release if that's possible. We'd really like this for the iceberg-rust encryption work
Reacted by Adam ReeveI plan to make an RC for 59 this week
I'd like to put forward #9683 as a requested item for this release.
It adds an MSD radix sort kernel (
arrow_row::radix::radix_sort_to_indices) that sorts directly on row-encoded keys, taking advantage of the row format already being big-endian and memcmp-comparable. It consistently beatslexsort_to_indicesfor multi-column sorts (up to ~2.5x at 32K rows), and the docs are upfront about where it loses.Worth landing because it's:
- Correct — 17 tests covering sort-option combinations, nulls, NaN/Inf, threshold boundaries, a fuzz test, and cross-validation against comparison sort on the same
Rows. - Well documented — clear guidance on when to use it vs
lexsort_to_indices, backed by benchmarks. - Foundational — operates on
Rows, returns indices, composes withtake. A building block for downstream sorting work, not a full sort-path replacement.
There's already downstream interest in building on it. I have a series of (now-closed) experimental DataFusion PRs exploring integration into
ExternalSorter:- perf: Bring over apache/arrow-rs/9683 radix sort, integrate into ExternalSorter datafusion#21525 — bring over the kernel, integrate into
ExternalSorter - perf: Optimize ExternalSorter with chunked sort pipeline and radix sort kernel datafusion#21600 — chunked sort pipeline + radix kernel
- perf: Coalesce batches before sorting in ExternalSorter to reduce merge fan-in datafusion#21629 — coalesce batches before sorting to reduce merge fan-in
- perf: another ExternalSorter refactor datafusion#21688 — further
ExternalSorterrefactor
These showed the kernel's utility but also that getting a real win needs a careful redesign of the sort stream rather than a drop-in swap. Landing the kernel in a release would give DataFusion (and other downstream consumers) something stable to build that work against.
CC @Dandandan
- Correct — 17 tests covering sort-option combinations, nulls, NaN/Inf, threshold boundaries, a fuzz test, and cross-validation against comparison sort on the same
- added a commit that references this issue
on Jun 2, 2026 3 remaining items
Ok, I think we have everything we planned for the release
Now I just need
- a committer to approve this PR Prepare for
59.0.0release #10063 - performance results from chore: Update to arrow/parquet 59.0.0 datafusion#22744 to confirm no regressions
And then I'll cut the RC
Reacted by Ed Seidl and Swanand Mulay- a committer to approve this PR Prepare for
- added a commit that references this issue
on Jun 4, 2026 I tried to make an RC (details below) but the verification script fails:
Emaikl
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 59.0.0. This release candidate is based on commit: 9f96a8f5c3f47c6303c7d17ac7f00ad4ad0df513 [1]. The SHA256 of the release candidate is: a22e39069a3bbcc02a9ba9837b36e6b87686c3bad18fe0a762aa4851f35499d8 The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. The release tracking issue is: [4] Please download, verify checksums and signatures, run the unit tests, and vote on the release. There is a script [4] that automates some of the verification. The vote will be open for at least 72 hours. [ ] +1 Release this as Apache Arrow Rust 59.0.0 [ ] +0 [ ] -1 Do not release this as Apache Arrow Rust 59.0.0 because... [1]: https://github.com/apache/arrow-rs/tree/9f96a8f5c3f47c6303c7d17ac7f00ad4ad0df513 [2]: https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-rs-59.0.0-rc1 [3]: https://github.com/apache/arrow-rs/blob/9f96a8f5c3f47c6303c7d17ac7f00ad4ad0df513/CHANGELOG.md [4]: https://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh [5]: https://github.com/apache/arrow-rs/issues/9110But the verification script fails like this:
failures: ---- bad_data::test_invalid_files stdout ---- thread 'bad_data::test_invalid_files' (96123527) panicked at parquet/tests/arrow_reader/bad_data.rs:68:9: Found new file in bad_data, please add test: ARROW-GH-47662.parquet note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace failures: bad_data::test_invalid_files test result: FAILED. 85 passed; 1 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.79s error: test failed, to rerun pass `-p parquet --test arrow_reader`I think we need to update the test. I'll see if I can figure out what to do in the morning (but I am basically out of time now)
Hmmm, I had to bump the parquet-testing commit to pull in the test file for IEEE754, and with that file came another failure file. But that PR wasn't committed...does the build process update submodules?
Anyway, I have a fix for that in #9619 I can split out and submit today.
Reacted by Andrew Lamb- added a commit that references this issue
on Jun 5, 2026 RC2 is out for voting
https://lists.apache.org/thread/xlozjylbqfo7tgh2lcvb6d3dvj5bwwxdThank you @etseidl
Reacted by Ed SeidlThe release was approved and is now available on crates.io
- https://lists.apache.org/thread/zmyp2zf4g3snxsc6nl977y6fm4g39stk
- https://crates.io/crates/arrow/59.0.0
The next release is tracked here
Thank you all for the help
Reacted by Adrian Garcia Badaracco and Matt Butrovich- added a commit that references this issue
on Jun 10, 2026 - added a commit that references this issue
on Jun 11, 2026
Let's keep the code flowing. Part of
Release Checklist:
59.0.0release #1006359.1.0(June 2026) #9878Required items for release
Requested items for release
take_bytesperf in the null cases between 10-25% #9625 by @AdamGS