Repository navigation
Release DataFusion 55.2.0 (patch) Release (Oct 2026) #25758
Description
Activity
I'd like to propose #25383 for 55.2.0. The backport is #25814.
On 55.1.0, a final aggregate that has spilled can fail with
ResourcesExhaustedwhile replaying its spill files. The spill merge reserves as much memory as the pool will give it, and the stream that aggregates the merged rows cannot spill. Comet hits this under memory pressure (apache/datafusion-comet#6254), and #25423 is the same failure inaggregate_memory_spill.slt.#25814 is stacked on #25805, the #24740 backport, because #25383 builds on #24740. So #25805 needs to go in first.
Reacted by xudong.w and Andrew Lamb- added a commit that references this issue
on Sep 29, 2026 I'd like to propose #24790 for 55.2.0. It fixes #24786: when a file has an INT96 column, the INT96 coercion rebuilds struct, list and map fields without their metadata, so Parquet field ids on those containers are lost while leaf ids survive. Comet always enables the coercion, so with
spark.sql.parquet.fieldId.read.enableda container whose id sits only on the container is null filled, while Spark reads it (apache/datafusion-comet#6131).The fix adds
.with_metadata(...)to the three container constructors inschema_coercion.rs, plus tests, and cherry-picks cleanly ontobranch-55. I can open the backport PR.BTW do we have a volunteer to help run this release?
BTW do we have a volunteer to help run this release?
I can take care of it
Reacted by Andrew Lamb- added a commit that references this issue
on Sep 30, 2026 @adriangb WDYT about porting #25838? seems like a good correctness patch
i’m a bit scared. it seems correct but also seems like the kind of thing that could cause issues (if eg even 1 query that used to be allowed and correct now is rejected). maybe we wait until someone (could be you) asks for the back port because they actually need it?
Reacted by Oleks VHi folks, I'm planning to create RC this weekend, let me know if you expecting backports, so I can review them
Reacted by Andrew Lamb- added a commit that references this issue
on Oct 1, 2026 Would it be possible to include #25922 ?
Reacted by Oleks VHi folks, I'm planning to create RC this weekend, let me know if you expecting backports, so I can review them
thanks -- let me know if I can do anything to help
Reacted by Oleks V- added a commit that references this issue
on Oct 4, 2026 - Reacted by Andrew Lamb
@alamb @timsaucer please grant me https://crates.io/crates/datafusion-proto-models , blocks 55.2.0 release
@alamb @timsaucer please grant me https://crates.io/crates/datafusion-proto-models , blocks 55.2.0 release
I sent you the invite. I'm checking who else is missing now.
Reacted by Oleks VThanks everyone the release is available https://dist.apache.org/repos/dist/release/datafusion/datafusion-55.2.0/ and
crates.ioartifacts publishedReacted by Viktor Yershov and Andrew Lamb- added a commit that references this issue
on Oct 7, 2026
Is your feature request related to a problem or challenge?
55.1.0(minor/patch) Release (Sep 2026) #24462Plan a 55.2.0 patch release to fix any bugs / issues found in 55.1.0.
Steps:
Potential Content:
Bug Fixes
EnsureRequirements, new in 55.0.0 via Add EnsureRequirements: merged EnforceDistribution + EnforceSorting with idempotent pushdown_sorts #21976, can silently drop a pushed-down fetch and return too many rows)RepartitionExecdeadlock under a memory limit; spill pool code changed in 55 via Resolve lost wakeup in SpillPoolReader with multiple concurrent SpillPoolWriters #23522)Performance
col = <literal>when statistics show the column is entirely NULL #24769 (@jensholdgaard); Fix: fix: prune row groups when file statistics collapse the predicate to a constant #24770 (@jensholdgaard); Backport: TBD (row group pruning lost forcol = <literal>on all-NULL columns; bisected to Remove redundantcollect_statandtarget_partitionsonListingOptions#22969 which shipped in 55.0.0)OrderedFinalAggregateStream#25639 (@2010YOUY01); Backport: TBDOther
output_batchesmetric does not match the number of emitted batches #24469 (@alamb); Fix: fix: count UnnestExec output batches after splitting #24482 (@goutamadwant); Backport: TBD (UnnestExecoutput_batchesmetric is wrong; introduced by fix: make UnnestExec respect datafusion.execution.batch_size #24384, which was backported to 55.1.0 as [branch-55] fix: make UnnestExec respect datafusion.execution.batch_size (#24384) #24529)datafusionin this scope" #24474 (@alamb); Backport: [branch-55] Reduce datafusion-spark dependency footprint (#24351) #25277 (allows publishingdatafusion-sparkwithout manual intervention)rustlsfix: update rustls to address RUSTSEC-2026-0285 #25309 (@Xuanwo); Backport: [branch-55] fix: update rustls to address RUSTSEC-2026-0285 (#25309) #25514datafusion-cliMinIO docker login failed #25215 (@comphead); Backport: [branch-55] Pull the MinIO test image from quay.io (#25092, #25216) #25485 (pull the MinIO test image from quay.io; CI only)Minor Features: