Repository navigation
feat!: Support compression codecs for Avro Files (including manifest and manifest lists) - #1851
emkornfield wants to merge 1 commit into
Conversation
|
looks like a clippy issue in CI |
Yeah working on a fix. |
|
@liurenjie1024 @kevinjqliu would you have time to review? |
kevinjqliu
left a comment
There was a problem hiding this comment.
Generally LGTM, I left a few comments around using the same string as the java implementation
| /// | ||
| /// # Compression Levels | ||
| /// | ||
| /// The compression level mapping is based on miniz_oxide's CompressionLevel enum: |
There was a problem hiding this comment.
clarify this is for gzip.
There was a problem hiding this comment.
nit: maybe merge this comment with the one above (L61-L66), the compression levels are explained twice
kevinjqliu
left a comment
There was a problem hiding this comment.
this looks great, thanks!
adding the "breaking" label since we're modifying a few public functions
|
@kevinjqliu thanks for the reviews, I also added more e2e tests for manifest and manifest list writers to confirm compression. |
I'm not sure that I can access labels as a contributor, I put a breaking change note in the PR description. |
liurenjie1024
left a comment
There was a problem hiding this comment.
Thanks @emkornfield for this pr. In general, I think this is a useful feature implementation. But we should not mix two things in one pr, I would suggest to split them into two prs: one for table metadata, and one for manifests.
| fn parse_optional_property<T: std::str::FromStr>( | ||
| properties: &HashMap<String, String>, | ||
| key: &str, | ||
| ) -> Result<Option<T>, anyhow::Error> |
There was a problem hiding this comment.
We should use this crate's error.
There was a problem hiding this comment.
Do we really need this method? I think passing a None to default value in parse_property would be enough?
There was a problem hiding this comment.
I'm new to rust but would that work? Doesn't parse_property, always return an extension of String instead of an Option? It is useful to know if the value was actually configured (and thus the option return)?
There was a problem hiding this comment.
I think in iceberg all table properties are optional, if they have default values, then we should return a default value, otherwise we should return an error.
There was a problem hiding this comment.
I'll see if I can make this work with empty string.
There was a problem hiding this comment.
I think in iceberg all table properties are optional, if they have default values, then we should return a default value, otherwise we should return an error.
The problem here is that the default value is dependent on the value for compression, it seems baking this into the parsing layer is coupling two concerns (business logic for parsing) and the actual values. To eliminate this method I've used a sentinel value for the old method. Let me know if that is seems OK.
|
|
||
| // Helper function to parse a property from a HashMap | ||
| // If the property is not found, use the default value | ||
| fn parse_property<T: std::str::FromStr>( |
There was a problem hiding this comment.
this is cleanup because it sounds like the crate's errors are preferred I can revert this.
|
@liurenjie1024 split of this PR to just be about Avro. Also based on feedback, I cleaned up some additional imports/Errors that weren't using the Crate's error before. |
kevinjqliu
left a comment
There was a problem hiding this comment.
This LGTM, a couple of nit comments
Thanks for adding the additional tests for manifest and manifest list writers
I also added the "breaking" label, forgot to after i commented 😄
| avro_compression_level: { | ||
| let level = parse_property( | ||
| props, | ||
| TableProperties::PROPERTY_AVRO_COMPRESSION_LEVEL, | ||
| 255u8, | ||
| )?; | ||
| if level == 255 { None } else { Some(level) } | ||
| }, |
There was a problem hiding this comment.
is this not valid here?
avro_compression_level: parse_property(
props,
TableProperties::PROPERTY_AVRO_COMPRESSION_LEVEL,
TableProperties::PROPERTY_AVRO_COMPRESSION_LEVEL_DEFAULT,
)?,
avro_compression_level and PROPERTY_AVRO_COMPRESSION_LEVEL_DEFAULT are both Option<u8>
There was a problem hiding this comment.
I might be misunderstanding the comment. But I should have cleaned up pub const PROPERTY_AVRO_COMPRESSION_LEVEL_DEFAULT: Option<u8> = None;, I don't think it is used anymore.
In a prior revision I added a method (parse_optional_property) that could return None value directly but @liurenjie1024 questioned whether this was useful.
| fn default() -> Self { | ||
| Self { | ||
| codec: TableProperties::PROPERTY_AVRO_COMPRESSION_CODEC_DEFAULT.to_string(), | ||
| level: None, |
There was a problem hiding this comment.
| level: None, | |
| level: TableProperties::PROPERTY_AVRO_COMPRESSION_LEVEL_DEFAULT, |
| /// | ||
| /// # Compression Levels | ||
| /// | ||
| /// The compression level mapping is based on miniz_oxide's CompressionLevel enum: |
There was a problem hiding this comment.
nit: maybe merge this comment with the one above (L61-L66), the compression levels are explained twice
| pub write_target_file_size_bytes: usize, | ||
| /// Compression codec for Avro files (manifests, manifest lists) | ||
| pub avro_compression_codec: String, | ||
| /// Compression level for Avro files (None uses codec-specific defaults: gzip=9, zstd=1) |
There was a problem hiding this comment.
| /// Compression level for Avro files (None uses codec-specific defaults: gzip=9, zstd=1) | |
| /// Compression level for Avro files (None uses codec-specific defaults) |
nit: these might change in the future, so i think its better to not specify the values here
there are a few other instances of this same comment
|
|
||
| /// Settings for compression codec and level. | ||
| #[derive(Debug, Clone, PartialEq, Eq)] | ||
| pub struct CompressionSettings { |
There was a problem hiding this comment.
I feel this struct is complete unnecessary. We should just add a avro_write_codec: Codec in TableProperties, and everything else should just use that field.
|
|
||
| // Use case-insensitive comparison to match Java implementation | ||
| match codec.map(|s| s.to_lowercase()).as_deref() { | ||
| Some("gzip") => { |
There was a problem hiding this comment.
Use constants rather than raw string.
| // Default level for gzip/deflate is 9 (BestCompression) to match Java | ||
| use miniz_oxide::deflate::CompressionLevel; | ||
|
|
||
| let compression_level = match level.unwrap_or(9) { |
| // Map compression level to miniz_oxide::deflate::CompressionLevel | ||
| // Reference: https://docs.rs/miniz_oxide/latest/miniz_oxide/deflate/enum.CompressionLevel.html | ||
| // Default level for gzip/deflate is 9 (BestCompression) to match Java | ||
| use miniz_oxide::deflate::CompressionLevel; |
There was a problem hiding this comment.
Why put importing here?
| /// The target file size for files. | ||
| pub write_target_file_size_bytes: usize, | ||
| /// Compression codec for Avro files (manifests, manifest lists) | ||
| pub avro_compression_codec: String, |
There was a problem hiding this comment.
| pub avro_compression_codec: String, | |
| pub avro_compression_codec: Codec, |
| /// Compression codec for Avro files (manifests, manifest lists) | ||
| pub avro_compression_codec: String, | ||
| /// Compression level for Avro files (None uses codec-specific defaults: gzip=9, zstd=1) | ||
| pub avro_compression_level: Option<u8>, |
| /// Compression level for Avro files | ||
| pub const PROPERTY_AVRO_COMPRESSION_LEVEL: &str = "write.avro.compression-level"; | ||
| /// Default Avro compression level (None, uses codec-specific defaults: gzip=9, zstd=1) | ||
| pub const PROPERTY_AVRO_COMPRESSION_LEVEL_DEFAULT: Option<u8> = None; |
There was a problem hiding this comment.
The spec says there is no default value, I don't think we need this one.
| [workspace.dependencies] | ||
| anyhow = "1.0.72" | ||
| apache-avro = { version = "0.20", features = ["zstandard"] } | ||
| apache-avro = { version = "0.20", features = ["zstandard", "snappy"] } |
There was a problem hiding this comment.
This enable the ability for avro to read/write snappy compressed files.
|
|
||
| #[tokio::test] | ||
| async fn test_manifest_list_writer_with_compression() { | ||
| use std::fs; |
There was a problem hiding this comment.
move these to top level.
| let level_raw = parse_property( | ||
| props, | ||
| TableProperties::PROPERTY_AVRO_COMPRESSION_LEVEL, | ||
| 255u8, |
There was a problem hiding this comment.
maybe make a constant?
|
Moving to draft until we finalize #1876 |
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
|
Still waiting on Moving to draft until we finalize #1876, I'll resolve conflicts. |
## Which issue does this PR close? Split off from #1851 - Partially fixes #1731. ## What changes are included in this PR? This change honors the compression setting for metadata.json file (`write.metadata.compression-codec`). ## Are these changes tested? Add unit test to verify files are gzipped when the flag is enabled. BREAKING CHANGE: Make `write_to` take `MetadataLocation` --------- Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io>
## Which issue does this PR close? This is an intermediate PR for #1731 I'm splitting out changes from #1851 to the compression codec to make it easier to review. Once we decide on approach here and merge it I'll update #1851 accordingly. ## What changes are included in this PR? - Add optional compression level to gzip and zstd (needed for when avro compression usage). - Add Snappy as a compression codec (also will be used for Avro) - Manually code up some previously auto-generated methods as a result. AI helped with an initial version of this PR. ## Are these changes tested? Additional unit tests
) ## Which issue does this PR close? Split off from apache#1851 - Partially fixes apache#1731. ## What changes are included in this PR? This change honors the compression setting for metadata.json file (`write.metadata.compression-codec`). ## Are these changes tested? Add unit test to verify files are gzipped when the flag is enabled. BREAKING CHANGE: Make `write_to` take `MetadataLocation` --------- Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io>
) ## Which issue does this PR close? Split off from apache#1851 - Partially fixes apache#1731. ## What changes are included in this PR? This change honors the compression setting for metadata.json file (`write.metadata.compression-codec`). ## Are these changes tested? Add unit test to verify files are gzipped when the flag is enabled. BREAKING CHANGE: Make `write_to` take `MetadataLocation` --------- Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> (cherry picked from commit b6de5db)
) ## Which issue does this PR close? Split off from apache#1851 - Partially fixes apache#1731. ## What changes are included in this PR? This change honors the compression setting for metadata.json file (`write.metadata.compression-codec`). ## Are these changes tested? Add unit test to verify files are gzipped when the flag is enabled. BREAKING CHANGE: Make `write_to` take `MetadataLocation` --------- Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> (cherry picked from commit b6de5db)
## Which issue does this PR close? This is an intermediate PR for apache#1731 I'm splitting out changes from apache#1851 to the compression codec to make it easier to review. Once we decide on approach here and merge it I'll update apache#1851 accordingly. ## What changes are included in this PR? - Add optional compression level to gzip and zstd (needed for when avro compression usage). - Add Snappy as a compression codec (also will be used for Avro) - Manually code up some previously auto-generated methods as a result. AI helped with an initial version of this PR. ## Are these changes tested? Additional unit tests (cherry picked from commit d8011a0)
* feat: Honor compression settings for metadata.json on write (#1876)
## Which issue does this PR close?
Split off from https://github.com/apache/iceberg-rust/pull/1851
- Partially fixes #1731.
## What changes are included in this PR?
This change honors the compression setting for metadata.json file
(`write.metadata.compression-codec`).
## Are these changes tested?
Add unit test to verify files are gzipped when the flag is enabled.
BREAKING CHANGE: Make `write_to` take `MetadataLocation`
---------
Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
* chore(deps): Bump quinn-proto from 0.11.13 to 0.11.14 in /bindings/python (#2228)
Bumps [quinn-proto](https://github.com/quinn-rs/quinn) from 0.11.13 to
0.11.14.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/quinn-rs/quinn/releases">quinn-proto's
releases</a>.</em></p>
<blockquote>
<h2>quinn-proto 0.11.14</h2>
<p><a href="https://github.com/jxs"><code>@jxs</code></a> reported a
denial of service issue in quinn-proto 5 days ago:</p>
<ul>
<li><a
href="https://github.com/quinn-rs/quinn/security/advisories/GHSA-6xvm-j4wr-6v98">https://github.com/quinn-rs/quinn/security/advisories/GHSA-6xvm-j4wr-6v98</a></li>
</ul>
<p>We coordinated with them to release this version to patch the issue.
Unfortunately the maintainers missed these issues during code review and
we did not have enough fuzzing coverage -- we regret the oversight and
have added an additional fuzzing target.</p>
<p>Organizations that want to participate in coordinated disclosure can
contact us privately to discuss terms.</p>
<h2>What's Changed</h2>
<ul>
<li>Fix over-permissive proto dependency edge by <a
href="https://github.com/Ralith"><code>@Ralith</code></a> in <a
href="https://redirect.github.com/quinn-rs/quinn/pull/2385">quinn-rs/quinn#2385</a></li>
<li>0.11.x: avoid unwrapping VarInt decoding during parameter parsing by
<a href="https://github.com/djc"><code>@djc</code></a> in <a
href="https://redirect.github.com/quinn-rs/quinn/pull/2559">quinn-rs/quinn#2559</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/quinn-rs/quinn/commit/2c315aa7f9c2a6c1db87f8f51f40623a427c78fd"><code>2c315aa</code></a>
proto: bump version to 0.11.14</li>
<li><a
href="https://github.com/quinn-rs/quinn/commit/8ad47f431e7deb82c08b09c2e33ef85aa88fd212"><code>8ad47f4</code></a>
Use newer rustls-pki-types PEM parser API</li>
<li><a
href="https://github.com/quinn-rs/quinn/commit/c81c0289abe30d8437ccbf9b6304e2bc9c707cea"><code>c81c028</code></a>
ci: fix workflow syntax</li>
<li><a
href="https://github.com/quinn-rs/quinn/commit/0050172969f7e69e136c433181330da7790d8d73"><code>0050172</code></a>
ci: pin wasm-bindgen-cli version</li>
<li><a
href="https://github.com/quinn-rs/quinn/commit/8a6f82c58d1c565eab78f986e614223e6ed76a85"><code>8a6f82c</code></a>
Take semver-compatible dependency updates</li>
<li><a
href="https://github.com/quinn-rs/quinn/commit/e52db4ad8df0f9720e7b0e32ecc0e48c9a93de0f"><code>e52db4a</code></a>
Apply suggestions from clippy 1.91</li>
<li><a
href="https://github.com/quinn-rs/quinn/commit/6df7275c582ca9b7225e0ccf9f9871a55eb73155"><code>6df7275</code></a>
chore: Fix <code>unnecessary_unwrap</code> clippy</li>
<li><a
href="https://github.com/quinn-rs/quinn/commit/c8eefa07e087b06d8f2b78ff262ce8ac952994f1"><code>c8eefa0</code></a>
proto: avoid unwrapping varint decoding during parameters parsing</li>
<li><a
href="https://github.com/quinn-rs/quinn/commit/9723a977754c8662001b0fef97aab8f3ddf1df92"><code>9723a97</code></a>
fuzz: add fuzzing target for parsing transport parameters</li>
<li><a
href="https://github.com/quinn-rs/quinn/commit/eaf0ef30252cef4acec21f150427e604cd4271c9"><code>eaf0ef3</code></a>
Fix over-permissive proto dependency edge (<a
href="https://redirect.github.com/quinn-rs/quinn/issues/2385">#2385</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/quinn-rs/quinn/compare/quinn-proto-0.11.13...quinn-proto-0.11.14">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/apache/iceberg-rust/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Add catalog test suite to unify catalog's behavior. (#2131)
## Which issue does this PR close?
- Closes #2086 .
## What changes are included in this PR?
In this pr we introduced catalog test suite in catalog-loader, which
could unify the behavior of catalogs.
## Are these changes tested?
Yes.
---------
Co-authored-by: Ray Liu <liurenjie2008@gmail.com>
* chore(deps): Bump datafusion from 52.2.0 to 52.3.0 (#2235)
Bumps [datafusion](https://github.com/apache/datafusion) from 52.2.0 to
52.3.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/apache/datafusion/commit/28d012a41a3017b5f682ef6b01468a7ff9a48fb7"><code>28d012a</code></a>
[branch-52] Bump to 52.3.0 and changelog (<a
href="https://redirect.github.com/apache/datafusion/issues/20790">#20790</a>)</li>
<li><a
href="https://github.com/apache/datafusion/commit/1bd7082b798d0d55c1e90c7be1d7e3dba057c288"><code>1bd7082</code></a>
[branch-52] Fix repartition from dropping data when spilling (<a
href="https://redirect.github.com/apache/datafusion/issues/20672">#20672</a>)
(<a
href="https://redirect.github.com/apache/datafusion/issues/20777">#20777</a>)</li>
<li><a
href="https://github.com/apache/datafusion/commit/9797095e152749721bec07c0944fe664acaa0849"><code>9797095</code></a>
[branch-52] perf: sort replace free()->try_grow() pattern with
try_resize() t...</li>
<li><a
href="https://github.com/apache/datafusion/commit/afc1c72a15bdd31e15a7e354e86a505be7882f08"><code>afc1c72</code></a>
[branch-52] FFI_TableOptions are using default values only (<a
href="https://redirect.github.com/apache/datafusion/issues/20705">#20705</a>)</li>
<li><a
href="https://github.com/apache/datafusion/commit/d317d00b886bbf11cb489e4c4bdc2280b3ca9e07"><code>d317d00</code></a>
[branch-52] fix: <code>HashJoin</code> panic with String dictionary keys
(don't flatten ...</li>
<li><a
href="https://github.com/apache/datafusion/commit/72ea8ec086e59220f6b255ea565e710990ad7967"><code>72ea8ec</code></a>
[branch-52] Fix constant value from stats (<a
href="https://redirect.github.com/apache/datafusion/issues/20042">#20042</a>)
(<a
href="https://redirect.github.com/apache/datafusion/issues/20709">#20709</a>)</li>
<li><a
href="https://github.com/apache/datafusion/commit/9a67de58c027e6057aa37327ae4d0192d5c45fc5"><code>9a67de5</code></a>
[branch-52] Fix Arrow Spill Underrun (<a
href="https://redirect.github.com/apache/datafusion/issues/20159">#20159</a>)
(<a
href="https://redirect.github.com/apache/datafusion/issues/20684">#20684</a>)</li>
<li><a
href="https://github.com/apache/datafusion/commit/19a0fcaa276c86beda544c6e01c75f6e0639767e"><code>19a0fca</code></a>
[branch-52] SortMergeJoin don't wait for all input before emitting (<a
href="https://redirect.github.com/apache/datafusion/issues/20699">#20699</a>)</li>
<li>See full diff in <a
href="https://github.com/apache/datafusion/compare/52.2.0...52.3.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump serde_with from 3.17.0 to 3.18.0 (#2233)
Bumps [serde_with](https://github.com/jonasbb/serde_with) from 3.17.0 to
3.18.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/jonasbb/serde_with/releases">serde_with's
releases</a>.</em></p>
<blockquote>
<h2>serde_with v3.18.0</h2>
<h3>Added</h3>
<ul>
<li>Support <code>OneOrMany</code> with more sequence and set types (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/929">#929</a>)</li>
</ul>
<h3>Changed</h3>
<ul>
<li>Bump MSRV to 1.88 due to the <code>darling</code> dependency</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/jonasbb/serde_with/commit/d50ec962c6ecad7d8972f95d7ee7cea398b7eb41"><code>d50ec96</code></a>
Bump version to 3.18.0 (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/931">#931</a>)</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/984fe3252ecd47526f452e39736f70f96b503f7c"><code>984fe32</code></a>
Bump version to 3.18.0</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/4ba41c70c7f12b2e543ae81480a50b4d76245419"><code>4ba41c7</code></a>
Bump actions/upload-artifact from 6 to 7 in the github-actions group (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/927">#927</a>)</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/8fb2468ce24e822fc29cd9aa8ebb3feb3ddf1eb3"><code>8fb2468</code></a>
Bump actions/upload-artifact from 6 to 7 in the github-actions
group</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/aec0a23c15943bc4ca82d329695fabefb2b19174"><code>aec0a23</code></a>
Bump MSRV to 1.88 (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/930">#930</a>)</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/25c15a2c5c53f8fa71af91d699877147568338b8"><code>25c15a2</code></a>
Update time dependency to 0.3.47</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/93bd3f4bebec516e5608a12f09ad1859cdced9a7"><code>93bd3f4</code></a>
Update test output after darling update</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/f825dbffb12dd758c80247259a271956f1c484b4"><code>f825dbf</code></a>
Upgrade darling to 0.23.0</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/65cbd738f090f25d89ec4b350501b4aa5b38bd9e"><code>65cbd73</code></a>
Bump MSRV to 1.88</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/daff02ea264c3136131bfcff079304714f359bd9"><code>daff02e</code></a>
Extend OneOrMany implementation to more collection types (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/929">#929</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/jonasbb/serde_with/compare/v3.17.0...v3.18.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump tempfile from 3.26.0 to 3.27.0 (#2234)
* chore(deps): Bump lz4_flex from 0.12.0 to 0.12.1 (#2239)
Bumps [lz4_flex](https://github.com/pseitz/lz4_flex) from 0.12.0 to
0.12.1.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/PSeitz/lz4_flex/blob/main/CHANGELOG.md">lz4_flex's
changelog</a>.</em></p>
<blockquote>
<h1>0.12.1 (2026-03-14)</h1>
<h3>Security Fix</h3>
<ul>
<li>Fix handling of invalid match offsets during decompression <a
href="https://github.com/PSeitz/lz4_flex/commit/a0b9154">#a0b9154</a>
(thanks <a
href="https://github.com/Marcono1234"><code>@Marcono1234</code></a>)</li>
</ul>
<pre><code>Invalid match offsets (offset == 0) during decompression were
not properly
handled, which could lead to invalid memory reads on untrusted input.
Users on 0.12.x should upgrade to 0.12.1.
</code></pre>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/PSeitz/lz4_flex/commit/fa48c987a88df5059a49fe7519c028d6f2b8caf4"><code>fa48c98</code></a>
bump version to 0.12.1</li>
<li><a
href="https://github.com/PSeitz/lz4_flex/commit/a0b9154becbe22da3ce91211d7b6619c289723cf"><code>a0b9154</code></a>
fix handling of invalid match offsets during decompression</li>
<li>See full diff in <a
href="https://github.com/pseitz/lz4_flex/compare/0.12.0...0.12.1">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/apache/iceberg-rust/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump lz4_flex from 0.12.0 to 0.12.1 in /bindings/python (#2238)
Bumps [lz4_flex](https://github.com/pseitz/lz4_flex) from 0.12.0 to
0.12.1.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/PSeitz/lz4_flex/blob/main/CHANGELOG.md">lz4_flex's
changelog</a>.</em></p>
<blockquote>
<h1>0.12.1 (2026-03-14)</h1>
<h3>Security Fix</h3>
<ul>
<li>Fix handling of invalid match offsets during decompression <a
href="https://github.com/PSeitz/lz4_flex/commit/a0b9154">#a0b9154</a>
(thanks <a
href="https://github.com/Marcono1234"><code>@Marcono1234</code></a>)</li>
</ul>
<pre><code>Invalid match offsets (offset == 0) during decompression were
not properly
handled, which could lead to invalid memory reads on untrusted input.
Users on 0.12.x should upgrade to 0.12.1.
</code></pre>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/PSeitz/lz4_flex/commit/fa48c987a88df5059a49fe7519c028d6f2b8caf4"><code>fa48c98</code></a>
bump version to 0.12.1</li>
<li><a
href="https://github.com/PSeitz/lz4_flex/commit/a0b9154becbe22da3ce91211d7b6619c289723cf"><code>a0b9154</code></a>
fix handling of invalid match offsets during decompression</li>
<li>See full diff in <a
href="https://github.com/pseitz/lz4_flex/compare/0.12.0...0.12.1">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/apache/iceberg-rust/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: blackmwk <liurenjie1024@outlook.com>
* feat(io): Add delete_stream to Storage trait (#2216)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #2065
## What changes are included in this PR?
- Add `delete_stream` to `Storage` trait to support batch delete
- Expose `delete_stream` in `FileIO` as well
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
Added uts
Addded integtests for opendal
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* feat(storage): implement opendal resolving storage (#2231)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #2210
## What changes are included in this PR?
- Add OpenDalResolvingStorage
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
Added a new test
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* doc: Update README.md to include more components (#2248)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #.
## What changes are included in this PR?
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* fix(python): use resolving storage for python binding (#2246)
* chore: Change publish parallism back to 1 (#2254)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- publish has to be done one by one, otherwise we may see failure like
this: https://github.com/apache/iceberg-rust/actions/runs/23260056698
## What changes are included in this PR?
- Change publish parallism back to 1
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* Infra: Remove GitHub Actions from Dependabot configuration (#2267)
Removed GitHub Actions dependency update configuration.
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #.
## What changes are included in this PR?
Related to https://github.com/apache/iceberg-python/issues/3186
Dont auto update since we now depend on github action being allowlisted
by asf-infra first,
https://github.com/apache/infrastructure-actions/blob/main/approved_patterns.yml
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* ci: pin third-party actions to Apache-approved SHAs (#2266)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #.
## What changes are included in this PR?
Pin `astral-sh/setup-uv` to commit SHAs from Apache's
[infrastructure-actions
allowlist](https://github.com/apache/infrastructure-actions/blob/07f5f9d2b05fe0ec9886e3ef0a9d79797817f0cb/approved_patterns.yml#L9)
Fixes https://github.com/apache/infrastructure-actions/issues/550
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* chore(deps): fix audit check and disable inherited aws sdk default features (#2274)
## Which issue does this PR close?
- Fix the audit check by updating `aws-lc-sys` and `rustls-webpki`.
- Avoid pulling both the legacy `rustls` / Hyper 0.14 stack and the
newer `default-https-client` stack through inherited AWS SDK defaults.
([AWS SDK
announcement](https://github.com/awslabs/aws-sdk-rust/discussions/1257))
## What changes are included in this PR?
- Bump to `aws-lc-sys>=0.39.0` and `rustls-webpki>=0.103.10` to pass
security audit.
- Disable inherited AWS SDK default features for `aws-sdk-glue` and
`aws-sdk-s3tables`
- Explicitly enable `default-https-client` and `rt-tokio`
- Bump the minimum `aws-sdk-glue` version to `1.85`, the first version
that provides `default-https-client`
## Are these changes tested?
---------
Co-authored-by: blackmwk <liurenjie1024@outlook.com>
* chore(deps): Bump minijinja from 2.17.1 to 2.18.0 (#2273)
Bumps [minijinja](https://github.com/mitsuhiko/minijinja) from 2.17.1 to
2.18.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/mitsuhiko/minijinja/blob/main/CHANGELOG.md">minijinja's
changelog</a>.</em></p>
<blockquote>
<h2>2.18.0</h2>
<ul>
<li>Added keyword argument support (<code>width</code>,
<code>first</code>, <code>blank</code>) to the <code>indent</code>
filter for Jinja2 compatibility in Rust and Go. <a
href="https://redirect.github.com/mitsuhiko/minijinja/issues/864">#864</a></li>
<li>Added support for dotted integer lookup (for example
<code>foo.0</code>) in Rust and Go for Jinja compatibility. <a
href="https://redirect.github.com/mitsuhiko/minijinja/issues/881">#881</a></li>
<li>Added support for dotted filter and test names (including <code>foo
. bar . baz</code>) for Jinja compatibility. <a
href="https://redirect.github.com/mitsuhiko/minijinja/issues/879">#879</a></li>
<li>Fixed string escape handling to preserve unknown escapes (such as
<code>\s</code>) for Jinja compatibility in Rust and Go. <a
href="https://redirect.github.com/mitsuhiko/minijinja/issues/880">#880</a></li>
<li>Improved generic performance across template parsing, compilation,
and rendering.</li>
<li>Fixed <code>minijinja-cabi</code> ownership and pointer-safety
issues that could leak <code>mj_value</code>
values on error paths.</li>
<li>Added high-priority <code>minijinja-cabi</code> APIs for
callback-based functions/filters/tests,
globals, loaders, path joining, auto-escape configuration, and fuel
limits.</li>
<li>Switched <code>minijinja-cabi</code> header maintenance to manual
source-based syncing and
removed cbindgen-based generation tooling.</li>
<li>Added lightweight C smoke tests for <code>minijinja-cabi</code> (via
<code>make -C minijinja-cabi test</code>)
with coverage across all exported C ABI functions, and wired them into
top-level
testing and CI.</li>
<li>Added <code>render_captured</code> and
<code>render_captured_to</code> methods on <code>Template</code> which
return a <code>Captured</code> type holding the rendered output and the
template state.</li>
<li>Added <code>into_output</code> method on <code>Captured</code> to
consume and return the output string.</li>
<li>Deprecated <code>render_and_return_state</code>,
<code>eval_to_state</code>, and <code>render_to_write</code>
in favor of the new <code>render_captured</code> /
<code>render_captured_to</code> / <code>Captured</code> API.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/92f114d1fd62525b2b4dc1adb77ae1e83c1214a9"><code>92f114d</code></a>
release 2.18.0</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/80d30a7526a0119981a1664fab8036b7e64c0d14"><code>80d30a7</code></a>
refactor(vendor): prune unused self_cell API surface</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/50ce37a18ad368f22b4c40ff2b3355895ff58556"><code>50ce37a</code></a>
fix: typos</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/24891e10c207846fa264c0f8eca930045bbb5fca"><code>24891e1</code></a>
feat(filters): add kwargs support to indent filter for Jinja2
parity</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/4cca670f8a346832771d2a567f778b5dc4058156"><code>4cca670</code></a>
refactor: deprecate render_to_write in favor of render_captured_to</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/ac88f8e619e0b7d5a4e23819ed5d2ebc046029c6"><code>ac88f8e</code></a>
fix: correct typo render_capturedd_to -> render_captured_to</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/710137b2626cfae81b1eb935ea4c9df2435c053d"><code>710137b</code></a>
chore: remove dead_code allow and unused MutBorrow from vendored
self_cell</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/39d00e61a9f7246b7015dcf655f11159cde1d8cd"><code>39d00e6</code></a>
feat: Added new capture methods for state</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/42b0d089333363b8bd667ec99ab67ff7977ef6d4"><code>42b0d08</code></a>
feat: vendor self_cell and make loader default</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/cc12ae0812b8d85dd5963cfa373971fb0b1ff6da"><code>cc12ae0</code></a>
fix: make cabi compatible with older rustc</li>
<li>Additional commits viewable in <a
href="https://github.com/mitsuhiko/minijinja/compare/minijinja-go/v2.17.1...minijinja-go/v2.18.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: blackmwk <liurenjie1024@outlook.com>
* chore(deps): Bump datafusion from 52.3.0 to 52.4.0 (#2271)
Bumps [datafusion](https://github.com/apache/datafusion) from 52.3.0 to
52.4.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/apache/datafusion/commit/e5bad58716cf74612ff3b245010411425063c3ec"><code>e5bad58</code></a>
[branch-52] Update version to 52.4.0 and update changelog (<a
href="https://redirect.github.com/apache/datafusion/issues/21004">#21004</a>)</li>
<li><a
href="https://github.com/apache/datafusion/commit/e034c6b0b103c674c4576644007b30480565bec3"><code>e034c6b</code></a>
[branch-52] Update to use lz4_flex 0.12.1 and quinn-proto 0.11.14 (<a
href="https://redirect.github.com/apache/datafusion/issues/21009">#21009</a>)</li>
<li><a
href="https://github.com/apache/datafusion/commit/664099b60640097a982e63174a96d8828fe1dc0d"><code>664099b</code></a>
[branch-52] fix: InList Dictionary filter pushdown type mismatch (<a
href="https://redirect.github.com/apache/datafusion/issues/20962">#20962</a>)
(<a
href="https://redirect.github.com/apache/datafusion/issues/2">#2</a>...</li>
<li><a
href="https://github.com/apache/datafusion/commit/74aaa65001afd7bc649f471bcf634d52744c46fd"><code>74aaa65</code></a>
[branch-52] chore: Ignore RUSTSEC-2024-0014 (<a
href="https://redirect.github.com/apache/datafusion/issues/20862">#20862</a>)
(<a
href="https://redirect.github.com/apache/datafusion/issues/21020">#21020</a>)</li>
<li><a
href="https://github.com/apache/datafusion/commit/5881edec5d937036891bbec9e7cb01837d9155a5"><code>5881ede</code></a>
[branch-52] fix: SanityCheckPlan error with window functions and NVL
filter (...</li>
<li><a
href="https://github.com/apache/datafusion/commit/7e20eb7ddb3acf8174af7adec52859e28333d570"><code>7e20eb7</code></a>
[branch-52] perf: Cache num_output_rows in sort merge join to avoid O(n)
reco...</li>
<li><a
href="https://github.com/apache/datafusion/commit/e5547e2772fbaed693e7472f38feab690a7fe3ef"><code>e5547e2</code></a>
[branch-52] Fix duplicate group keys after hash aggregation spill (<a
href="https://redirect.github.com/apache/datafusion/issues/20724">#20724</a>)
(#...</li>
<li><a
href="https://github.com/apache/datafusion/commit/2947378e9ef9dbdda75b4ff047edcfc1a06ef0d2"><code>2947378</code></a>
[branch-52] fix: disable dynamic filter pushdown for non min/max
aggregates (...</li>
<li><a
href="https://github.com/apache/datafusion/commit/41acbf8e4bb4ac15003bd5365661e6b17551f7f0"><code>41acbf8</code></a>
[branch-52] fix: Return <code>probe_side.len()</code> for RightMark/Anti
count(*) querie...</li>
<li><a
href="https://github.com/apache/datafusion/commit/a5f6fbb4cd89a47e1036986abe201def15542093"><code>a5f6fbb</code></a>
[branch-52] fix: interval analysis error when have two filterexec that
inner ...</li>
<li>Additional commits viewable in <a
href="https://github.com/apache/datafusion/compare/52.3.0...52.4.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: blackmwk <liurenjie1024@outlook.com>
* chore(deps): Bump datafusion-cli from 52.2.0 to 52.3.0 (#2272)
* chore(deps): Bump datafusion-sqllogictest from 52.2.0 to 52.3.0 (#2270)
* fix: explicitly set Python interpreter in maturin build to prevent using wrong version (#2277)
* chore(deps): Bump rustls-webpki from 0.103.7 to 0.103.10 in bindings/python (#2278)
Addresses the security advisory GHSA-pwjx-qhcg-rvj4 for rustls-webpki <
0.103.10 in the Python bindings lockfile.
This is a rebase of #2268 onto main which already includes the root
Cargo.lock audit fix from #2274 (aws-lc-sys >= 0.39.0).
## Which issue does this PR close?
- Closes #.
## What changes are included in this PR?
## Are these changes tested?
ci.
* chore(deps): Bump bytes from 1.11.0 to 1.11.1 in /bindings/python (#2281)
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.11.0 to 1.11.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/tokio-rs/bytes/releases">bytes's
releases</a>.</em></p>
<blockquote>
<h2>Bytes v1.11.1</h2>
<h1>1.11.1 (February 3rd, 2026)</h1>
<ul>
<li>Fix integer overflow in <code>BytesMut::reserve</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md">bytes's
changelog</a>.</em></p>
<blockquote>
<h1>1.11.1 (February 3rd, 2026)</h1>
<ul>
<li>Fix integer overflow in <code>BytesMut::reserve</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/tokio-rs/bytes/commit/417dccdeff249e0c011327de7d92e0d6fbe7cc43"><code>417dccd</code></a>
Release bytes v1.11.1 (<a
href="https://redirect.github.com/tokio-rs/bytes/issues/820">#820</a>)</li>
<li><a
href="https://github.com/tokio-rs/bytes/commit/d0293b0e35838123c51ca5dfdf468ecafee4398f"><code>d0293b0</code></a>
Merge commit from fork</li>
<li>See full diff in <a
href="https://github.com/tokio-rs/bytes/compare/v1.11.0...v1.11.1">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/apache/iceberg-rust/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump time from 0.3.44 to 0.3.47 in /bindings/python (#2282)
Bumps [time](https://github.com/time-rs/time) from 0.3.44 to 0.3.47.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/time-rs/time/releases">time's
releases</a>.</em></p>
<blockquote>
<h2>v0.3.47</h2>
<p>See the <a
href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">changelog</a>
for details.</p>
<h2>v0.3.46</h2>
<p>See the <a
href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">changelog</a>
for details.</p>
<h2>v0.3.45</h2>
<p>See the <a
href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">changelog</a>
for details.</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">time's
changelog</a>.</em></p>
<blockquote>
<h2>0.3.47 [2026-02-05]</h2>
<h3>Security</h3>
<ul>
<li>
<p>The possibility of a stack exhaustion denial of service attack when
parsing RFC 2822 has been
eliminated. Previously, it was possible to craft input that would cause
unbounded recursion. Now,
the depth of the recursion is tracked, causing an error to be returned
if it exceeds a reasonable
limit.</p>
<p>This attack vector requires parsing user-provided input, with any
type, using the RFC 2822 format.</p>
</li>
</ul>
<h3>Compatibility</h3>
<ul>
<li>Attempting to format a value with a well-known format (i.e. RFC
3339, RFC 2822, or ISO 8601) will
error at compile time if the type being formatted does not provide
sufficient information. This
would previously fail at runtime. Similarly, attempting to format a
value with ISO 8601 that is
only configured for parsing (i.e. <code>Iso8601::PARSING</code>) will
error at compile time.</li>
</ul>
<h3>Added</h3>
<ul>
<li>Builder methods for format description modifiers, eliminating the
need for verbose initialization
when done manually.</li>
<li><code>date!(2026-W01-2)</code> is now supported. Previously, a space
was required between <code>W</code> and <code>01</code>.</li>
<li><code>[end]</code> now has a <code>trailing_input</code> modifier
which can either be <code>prohibit</code> (the default) or
<code>discard</code>. When it is <code>discard</code>, all remaining
input is ignored. Note that if there are components
after <code>[end]</code>, they will still attempt to be parsed, likely
resulting in an error.</li>
</ul>
<h3>Changed</h3>
<ul>
<li>More performance gains when parsing.</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>If manually formatting a value, the number of bytes written was one
short for some components.
This has been fixed such that the number of bytes written is always
correct.</li>
<li>The possibility of integer overflow when parsing an owned format
description has been effectively
eliminated. This would previously wrap when overflow checks were
disabled. Instead of storing the
depth as <code>u8</code>, it is stored as <code>u32</code>. This would
require multiple gigabytes of nested input to
overflow, at which point we've got other problems and trivial
mitigations are available by
downstream users.</li>
</ul>
<h2>0.3.46 [2026-01-23]</h2>
<h3>Added</h3>
<ul>
<li>All possible panics are now documented for the relevant
methods.</li>
<li>The need to use <code>#[serde(default)]</code> when using custom
<code>serde</code> formats is documented. This applies
only when deserializing an <code>Option<T></code>.</li>
<li><code>Duration::nanoseconds_i128</code> has been made public,
mirroring
<code>std::time::Duration::from_nanos_u128</code>.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/time-rs/time/commit/d5144cd2874862d46466c900910cd8577d066019"><code>d5144cd</code></a>
v0.3.47 release</li>
<li><a
href="https://github.com/time-rs/time/commit/f6206b050fd54817d8872834b4d61f605570e89b"><code>f6206b0</code></a>
Guard against integer overflow in release mode</li>
<li><a
href="https://github.com/time-rs/time/commit/1c63dc7985b8fa26bd8c689423cc56b7a03841ee"><code>1c63dc7</code></a>
Avoid denial of service when parsing Rfc2822</li>
<li><a
href="https://github.com/time-rs/time/commit/5940df6e72efb63d246ca1ca59a0f836ad32ad8a"><code>5940df6</code></a>
Add builder methods to avoid verbose construction</li>
<li><a
href="https://github.com/time-rs/time/commit/00881a4da1bc5a6cb6313052e5017dbd7daa40f0"><code>00881a4</code></a>
Manually format macros everywhere</li>
<li><a
href="https://github.com/time-rs/time/commit/bb723b6d826e46c174d75cd08987061984b0ceb7"><code>bb723b6</code></a>
Add <code>trailing_input</code> modifier to <code>end</code></li>
<li><a
href="https://github.com/time-rs/time/commit/31c4f8e0b56e6ae24fe0d6ef0e492b6741dda783"><code>31c4f8e</code></a>
Permit <code>W12</code> in <code>date!</code> macro</li>
<li><a
href="https://github.com/time-rs/time/commit/490a17bf306576850f33a86d3ca95d96db7b1dcd"><code>490a17b</code></a>
Mark error paths in well-known formats as cold</li>
<li><a
href="https://github.com/time-rs/time/commit/6cb1896a600be1538ecfab8f233fe9cfe9fa8951"><code>6cb1896</code></a>
Optimize <code>Rfc2822</code> parsing</li>
<li><a
href="https://github.com/time-rs/time/commit/6d264d59c25e3da0453c3defebf4640b0086a006"><code>6d264d5</code></a>
Remove erroneous <code>#[inline(never)]</code> attributes</li>
<li>Additional commits viewable in <a
href="https://github.com/time-rs/time/compare/v0.3.44...v0.3.47">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/apache/iceberg-rust/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* docs: Add DataFusion Comet to Users section of README (#2283)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- N/A.
## What changes are included in this PR?
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
- Add DataFusion Comet to the list of users with a description.
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
- N/A.
* feat(encryption) [1/N] Support encryption: Add crypto for AES-GCM (#2026)
Add Core Encryption Primitives for Iceberg Encryption Support.
Part of https://github.com/apache/iceberg-rust/issues/2034
## Summary
This PR introduces the foundational cryptographic primitives needed for
implementing encryption in iceberg-rust, providing AES-GCM encryption
operations that match the Java implementation's behavior and data
format.
## Motivation
Iceberg's Java implementation supports table-level encryption to protect
sensitive data at rest. To achieve feature parity and ensure
interoperability between Java and Rust implementations, we need to build
encryption support from the ground up. This PR provides the core
cryptographic operations that will serve as the foundation for the
complete encryption feature.
## Changes
New Module: encryption
Added a new encryption module with core AES-GCM cryptographic
operations:
- encryption/crypto.rs - Core encryption implementation
- EncryptionAlgorithm enum supporting AES-128-GCM as this is the only
algorithm currently supported in arrow parquet
- SecureKey struct with automatic memory zeroization for security
- AesGcmEncryptor providing encrypt/decrypt operations with AAD support
Key Features
1. Java-Compatible Format: Ciphertext format matches Java's
implementation exactly:
[12-byte nonce][encrypted data][16-byte GCM authentication tag]
1. This ensures files encrypted by Java can be decrypted by Rust and
vice versa.
2. Secure Key Handling: Uses the zeroize crate to automatically clear
encryption keys from memory when dropped, preventing key material from
lingering in memory.
3. Additional Authenticated Data (AAD): Full support for AAD to ensure
integrity of associated metadata that isn't encrypted.
4. Comprehensive Testing: 8 tests covering:
- Round-trip encryption/decryption for both AES-128 and AES-256
- AAD validation
- Empty plaintext handling
- Tamper detection
- Format compatibility verification
Dependencies Added
- aes-gcm = "0.10" - Industry-standard AES-GCM implementation
- zeroize = "1.7" - Secure memory cleanup for encryption keys
Compatibility
This implementation directly corresponds to Java's
https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/encryption/Ciphers.java:
| Java Class | Rust Implementation |
|-----------------------------|------------------------------------------|
| Ciphers.AesGcmEncryptor | AesGcmEncryptor::encrypt() |
| Ciphers.AesGcmDecryptor | AesGcmEncryptor::decrypt() |
| EncryptionAlgorithm.AES_GCM | EncryptionAlgorithm::Aes128Gcm|
Testing
Future Work
This PR is the first in a series to implement full encryption support.
Upcoming PRs will add:
1. Table properties for encryption configuration
2. Key management interfaces (KeyManagementClient trait)
3. EncryptionManager implementation
4. Native Parquet encryption integration
5. AWS KMS support
6. Integration with Table and FileIO
Review Notes
- This PR is intentionally minimal and self-contained
- No existing code paths are modified - this is purely additive
- The module is public but won't be used until future PRs wire it up
- Format compatibility with Java has been prioritized to ensure
interoperability
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #. https://github.com/apache/iceberg-rust/issues/2035
## What changes are included in this PR?
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
Yes
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* feat!(catalog): adding support for purge_table (#2232)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #2133
## What changes are included in this PR?
- Add catalog/utils.rs to provide helpers to delete table data using
file_io and table_metadata
- Add new API `purge_table` to `Catalog` trait and add default
implementation
- Implement purge_table for S3TableCatalog and RestCatalog
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
Added new tests in table_suite
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* deps: upgrade DataFusion to 53.0, Arrow to 58 (#2206)
## Which issue does this PR close?
- Closes #.
## What changes are included in this PR?
- Bump DataFusion to 53.0.0, Arrow/Parquet to 58, sqllogictest to 0.29,
pyo3 to 0.28.
- Adapt to DataFusion 53 API changes in physical plan executors and
python bindings.
- Update SLT expected test output.
## Are these changes tested?
Existing tests.
---------
Co-authored-by: Xander <zander181@googlemail.com>
* chore(deps): Bump uuid from 1.22.0 to 1.23.0 (#2291)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.22.0 to 1.23.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/uuid-rs/uuid/releases">uuid's
releases</a>.</em></p>
<blockquote>
<h2>v1.23.0</h2>
<h2>What's Changed</h2>
<ul>
<li>feat: add support for 'hyphenated' format in the serde module by <a
href="https://github.com/FrenchDilettante"><code>@FrenchDilettante</code></a>
in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/865">uuid-rs/uuid#865</a></li>
<li>Fix a number of bugs in time-related code by <a
href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/872">uuid-rs/uuid#872</a></li>
<li>Reword invalid char error message by <a
href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/873">uuid-rs/uuid#873</a></li>
<li>Impl cleanups by <a
href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/874">uuid-rs/uuid#874</a></li>
<li>Use LazyLock to synchronize v1/v6 context initialization by <a
href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/875">uuid-rs/uuid#875</a></li>
<li>Prepare for 1.23.0 release by <a
href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/876">uuid-rs/uuid#876</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/FrenchDilettante"><code>@FrenchDilettante</code></a>
made their first contribution in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/865">uuid-rs/uuid#865</a></li>
</ul>
<h2>Special thanks</h2>
<p><a href="https://github.com/meng-xu-cs"><code>@meng-xu-cs</code></a>
raised a series of bugs against the timestamp logic in <code>uuid</code>
using automated tooling. The issues themselves were reasonably and
responsibly presented and the end result is a better <code>uuid</code>
library for everyone. Thanks!</p>
<h1>Deprecations</h1>
<p>This release includes the following deprecations:</p>
<ul>
<li><code>Context</code>: Renamed to <code>ContextV1</code></li>
<li><code>Timestamp::from_gregorian</code>: Renamed to
<code>Timestamp::from_gregorian_time</code></li>
</ul>
<h1>Change to <code>Version::Max</code></h1>
<p><code>Version::Max</code>'s <code>u8</code> representation has
changed from <code>0xff</code> to <code>0x0f</code> to match the value
returned by <code>Uuid::get_version_num</code>.</p>
<h1>Change to <code>Uuid::get_version</code> for the max UUID</h1>
<p><code>Uuid::get_version</code> will only return
<code>Some(Version::Max)</code> if the UUID is actually the max UUID
(all bytes are <code>0xff</code>). Previously it would return
<code>Some</code> if only the version field was <code>0x0f</code>. This
change matches the behaviour of the nil UUID, which only returns
<code>Some(Version::Nil)</code> if the UUID is the nil UUID (all bytes
are <code>0x00</code>).</p>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/uuid-rs/uuid/compare/v1.22.0...v1.23.0">https://github.com/uuid-rs/uuid/compare/v1.22.0...v1.23.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/uuid-rs/uuid/commit/00ab922d5351607dfff520f37eb49cb9854fda73"><code>00ab922</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/876">#876</a> from
uuid-rs/cargo/v1.23.0</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/726ba45fe3491bf6253173d0be6b99ed3b1cbbb9"><code>726ba45</code></a>
prepare for 1.23.0 release</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/996dadea029e3976f52cba58e5e9b9a08c4f82c4"><code>996dade</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/875">#875</a> from
uuid-rs/fix/context-ordering</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/e14047993bc5a6180a96119436a983c19d79b084"><code>e140479</code></a>
simplify a use stmt</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/8ed9142847a22bc7707794bfee6b2016d4470772"><code>8ed9142</code></a>
reorganize and document more v7 context internals</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/e09a3225a8d99c5eadcbbeb7432195b2ea5ece76"><code>e09a322</code></a>
use LazyLock to synchronize v1/v6 context initialization</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/0f260cc67135ac20d914e387a47e59960247fdee"><code>0f260cc</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/874">#874</a> from
uuid-rs/chore/impl-cleanups</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/1419e91097fcffc7afa8f54eb41fdc912200b540"><code>1419e91</code></a>
clean up and refactor main lib tests</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/ceeaf4b7b59895497c59acdaf286233b1e7cc576"><code>ceeaf4b</code></a>
ensure we don't overflow on counters less than 12</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/63bc8f52e5042b9c729fa0380b9948b49fe397cc"><code>63bc8f5</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/873">#873</a> from
uuid-rs/fix/error-msg</li>
<li>Additional commits viewable in <a
href="https://github.com/uuid-rs/uuid/compare/v1.22.0...v1.23.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* ci: improve github workflows (#2289)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #.
## What changes are included in this PR?
Relates to https://github.com/apache/iceberg/issues/15742
This PR
- Add "ASF allowlist check"
- Pin commit for codeql.yml (zizmor recommended)
- Add back Github Action auto-update for dependabot (reverts #2267)
- Add cooldown to dependabot (zizmor recommended)
- `Swatinem/rust-cache@v2` -> `swatinem/rust-cache@v2` (fix case
sensitivity) [asf infra allowlist uses
lowercase](https://github.com/apache/infrastructure-actions/blob/fae466bc0d9821859a623cbc7648c750ff359ec6/approved_patterns.yml#L271)
We can add back dependabot for github action because the "ASF allowlist
check" will now alert when an action is not allowed (failures will no
longer be silent)
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* Make `convert_filters_to_predicate` public (#2118)
## What changes are included in this PR?
- Make `convert_filters_to_predicate` public in the DataFusion
integration to allow external usage of the filter conversion logic.
## Are these changes tested?
- This is a visibility change (`pub use`) and does not introduce new
logic.
Co-authored-by: Denis Semenov <d.s.semenov@vkteam.ru>
* ci: fix zizmor security f…
|
Is this work still active? |
|
Yes, sorry I've been delayed in getting this ready for a re-review. |
5df5513 to
ef52dde
Compare
* doc: Update README.md to include more components (#2248)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #.
## What changes are included in this PR?
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* fix(python): use resolving storage for python binding (#2246)
* chore: Change publish parallism back to 1 (#2254)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- publish has to be done one by one, otherwise we may see failure like
this: https://github.com/apache/iceberg-rust/actions/runs/23260056698
## What changes are included in this PR?
- Change publish parallism back to 1
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* Infra: Remove GitHub Actions from Dependabot configuration (#2267)
Removed GitHub Actions dependency update configuration.
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #.
## What changes are included in this PR?
Related to https://github.com/apache/iceberg-python/issues/3186
Dont auto update since we now depend on github action being allowlisted
by asf-infra first,
https://github.com/apache/infrastructure-actions/blob/main/approved_patterns.yml
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* ci: pin third-party actions to Apache-approved SHAs (#2266)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #.
## What changes are included in this PR?
Pin `astral-sh/setup-uv` to commit SHAs from Apache's
[infrastructure-actions
allowlist](https://github.com/apache/infrastructure-actions/blob/07f5f9d2b05fe0ec9886e3ef0a9d79797817f0cb/approved_patterns.yml#L9)
Fixes https://github.com/apache/infrastructure-actions/issues/550
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* chore(deps): fix audit check and disable inherited aws sdk default features (#2274)
## Which issue does this PR close?
- Fix the audit check by updating `aws-lc-sys` and `rustls-webpki`.
- Avoid pulling both the legacy `rustls` / Hyper 0.14 stack and the
newer `default-https-client` stack through inherited AWS SDK defaults.
([AWS SDK
announcement](https://github.com/awslabs/aws-sdk-rust/discussions/1257))
## What changes are included in this PR?
- Bump to `aws-lc-sys>=0.39.0` and `rustls-webpki>=0.103.10` to pass
security audit.
- Disable inherited AWS SDK default features for `aws-sdk-glue` and
`aws-sdk-s3tables`
- Explicitly enable `default-https-client` and `rt-tokio`
- Bump the minimum `aws-sdk-glue` version to `1.85`, the first version
that provides `default-https-client`
## Are these changes tested?
---------
Co-authored-by: blackmwk <liurenjie1024@outlook.com>
* chore(deps): Bump minijinja from 2.17.1 to 2.18.0 (#2273)
Bumps [minijinja](https://github.com/mitsuhiko/minijinja) from 2.17.1 to
2.18.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/mitsuhiko/minijinja/blob/main/CHANGELOG.md">minijinja's
changelog</a>.</em></p>
<blockquote>
<h2>2.18.0</h2>
<ul>
<li>Added keyword argument support (<code>width</code>,
<code>first</code>, <code>blank</code>) to the <code>indent</code>
filter for Jinja2 compatibility in Rust and Go. <a
href="https://redirect.github.com/mitsuhiko/minijinja/issues/864">#864</a></li>
<li>Added support for dotted integer lookup (for example
<code>foo.0</code>) in Rust and Go for Jinja compatibility. <a
href="https://redirect.github.com/mitsuhiko/minijinja/issues/881">#881</a></li>
<li>Added support for dotted filter and test names (including <code>foo
. bar . baz</code>) for Jinja compatibility. <a
href="https://redirect.github.com/mitsuhiko/minijinja/issues/879">#879</a></li>
<li>Fixed string escape handling to preserve unknown escapes (such as
<code>\s</code>) for Jinja compatibility in Rust and Go. <a
href="https://redirect.github.com/mitsuhiko/minijinja/issues/880">#880</a></li>
<li>Improved generic performance across template parsing, compilation,
and rendering.</li>
<li>Fixed <code>minijinja-cabi</code> ownership and pointer-safety
issues that could leak <code>mj_value</code>
values on error paths.</li>
<li>Added high-priority <code>minijinja-cabi</code> APIs for
callback-based functions/filters/tests,
globals, loaders, path joining, auto-escape configuration, and fuel
limits.</li>
<li>Switched <code>minijinja-cabi</code> header maintenance to manual
source-based syncing and
removed cbindgen-based generation tooling.</li>
<li>Added lightweight C smoke tests for <code>minijinja-cabi</code> (via
<code>make -C minijinja-cabi test</code>)
with coverage across all exported C ABI functions, and wired them into
top-level
testing and CI.</li>
<li>Added <code>render_captured</code> and
<code>render_captured_to</code> methods on <code>Template</code> which
return a <code>Captured</code> type holding the rendered output and the
template state.</li>
<li>Added <code>into_output</code> method on <code>Captured</code> to
consume and return the output string.</li>
<li>Deprecated <code>render_and_return_state</code>,
<code>eval_to_state</code>, and <code>render_to_write</code>
in favor of the new <code>render_captured</code> /
<code>render_captured_to</code> / <code>Captured</code> API.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/92f114d1fd62525b2b4dc1adb77ae1e83c1214a9"><code>92f114d</code></a>
release 2.18.0</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/80d30a7526a0119981a1664fab8036b7e64c0d14"><code>80d30a7</code></a>
refactor(vendor): prune unused self_cell API surface</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/50ce37a18ad368f22b4c40ff2b3355895ff58556"><code>50ce37a</code></a>
fix: typos</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/24891e10c207846fa264c0f8eca930045bbb5fca"><code>24891e1</code></a>
feat(filters): add kwargs support to indent filter for Jinja2
parity</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/4cca670f8a346832771d2a567f778b5dc4058156"><code>4cca670</code></a>
refactor: deprecate render_to_write in favor of render_captured_to</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/ac88f8e619e0b7d5a4e23819ed5d2ebc046029c6"><code>ac88f8e</code></a>
fix: correct typo render_capturedd_to -> render_captured_to</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/710137b2626cfae81b1eb935ea4c9df2435c053d"><code>710137b</code></a>
chore: remove dead_code allow and unused MutBorrow from vendored
self_cell</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/39d00e61a9f7246b7015dcf655f11159cde1d8cd"><code>39d00e6</code></a>
feat: Added new capture methods for state</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/42b0d089333363b8bd667ec99ab67ff7977ef6d4"><code>42b0d08</code></a>
feat: vendor self_cell and make loader default</li>
<li><a
href="https://github.com/mitsuhiko/minijinja/commit/cc12ae0812b8d85dd5963cfa373971fb0b1ff6da"><code>cc12ae0</code></a>
fix: make cabi compatible with older rustc</li>
<li>Additional commits viewable in <a
href="https://github.com/mitsuhiko/minijinja/compare/minijinja-go/v2.17.1...minijinja-go/v2.18.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: blackmwk <liurenjie1024@outlook.com>
* chore(deps): Bump datafusion from 52.3.0 to 52.4.0 (#2271)
Bumps [datafusion](https://github.com/apache/datafusion) from 52.3.0 to
52.4.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/apache/datafusion/commit/e5bad58716cf74612ff3b245010411425063c3ec"><code>e5bad58</code></a>
[branch-52] Update version to 52.4.0 and update changelog (<a
href="https://redirect.github.com/apache/datafusion/issues/21004">#21004</a>)</li>
<li><a
href="https://github.com/apache/datafusion/commit/e034c6b0b103c674c4576644007b30480565bec3"><code>e034c6b</code></a>
[branch-52] Update to use lz4_flex 0.12.1 and quinn-proto 0.11.14 (<a
href="https://redirect.github.com/apache/datafusion/issues/21009">#21009</a>)</li>
<li><a
href="https://github.com/apache/datafusion/commit/664099b60640097a982e63174a96d8828fe1dc0d"><code>664099b</code></a>
[branch-52] fix: InList Dictionary filter pushdown type mismatch (<a
href="https://redirect.github.com/apache/datafusion/issues/20962">#20962</a>)
(<a
href="https://redirect.github.com/apache/datafusion/issues/2">#2</a>...</li>
<li><a
href="https://github.com/apache/datafusion/commit/74aaa65001afd7bc649f471bcf634d52744c46fd"><code>74aaa65</code></a>
[branch-52] chore: Ignore RUSTSEC-2024-0014 (<a
href="https://redirect.github.com/apache/datafusion/issues/20862">#20862</a>)
(<a
href="https://redirect.github.com/apache/datafusion/issues/21020">#21020</a>)</li>
<li><a
href="https://github.com/apache/datafusion/commit/5881edec5d937036891bbec9e7cb01837d9155a5"><code>5881ede</code></a>
[branch-52] fix: SanityCheckPlan error with window functions and NVL
filter (...</li>
<li><a
href="https://github.com/apache/datafusion/commit/7e20eb7ddb3acf8174af7adec52859e28333d570"><code>7e20eb7</code></a>
[branch-52] perf: Cache num_output_rows in sort merge join to avoid O(n)
reco...</li>
<li><a
href="https://github.com/apache/datafusion/commit/e5547e2772fbaed693e7472f38feab690a7fe3ef"><code>e5547e2</code></a>
[branch-52] Fix duplicate group keys after hash aggregation spill (<a
href="https://redirect.github.com/apache/datafusion/issues/20724">#20724</a>)
(#...</li>
<li><a
href="https://github.com/apache/datafusion/commit/2947378e9ef9dbdda75b4ff047edcfc1a06ef0d2"><code>2947378</code></a>
[branch-52] fix: disable dynamic filter pushdown for non min/max
aggregates (...</li>
<li><a
href="https://github.com/apache/datafusion/commit/41acbf8e4bb4ac15003bd5365661e6b17551f7f0"><code>41acbf8</code></a>
[branch-52] fix: Return <code>probe_side.len()</code> for RightMark/Anti
count(*) querie...</li>
<li><a
href="https://github.com/apache/datafusion/commit/a5f6fbb4cd89a47e1036986abe201def15542093"><code>a5f6fbb</code></a>
[branch-52] fix: interval analysis error when have two filterexec that
inner ...</li>
<li>Additional commits viewable in <a
href="https://github.com/apache/datafusion/compare/52.3.0...52.4.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: blackmwk <liurenjie1024@outlook.com>
* chore(deps): Bump datafusion-cli from 52.2.0 to 52.3.0 (#2272)
* chore(deps): Bump datafusion-sqllogictest from 52.2.0 to 52.3.0 (#2270)
* fix: explicitly set Python interpreter in maturin build to prevent using wrong version (#2277)
* chore(deps): Bump rustls-webpki from 0.103.7 to 0.103.10 in bindings/python (#2278)
Addresses the security advisory GHSA-pwjx-qhcg-rvj4 for rustls-webpki <
0.103.10 in the Python bindings lockfile.
This is a rebase of #2268 onto main which already includes the root
Cargo.lock audit fix from #2274 (aws-lc-sys >= 0.39.0).
## Which issue does this PR close?
- Closes #.
## What changes are included in this PR?
## Are these changes tested?
ci.
* chore(deps): Bump bytes from 1.11.0 to 1.11.1 in /bindings/python (#2281)
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.11.0 to 1.11.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/tokio-rs/bytes/releases">bytes's
releases</a>.</em></p>
<blockquote>
<h2>Bytes v1.11.1</h2>
<h1>1.11.1 (February 3rd, 2026)</h1>
<ul>
<li>Fix integer overflow in <code>BytesMut::reserve</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md">bytes's
changelog</a>.</em></p>
<blockquote>
<h1>1.11.1 (February 3rd, 2026)</h1>
<ul>
<li>Fix integer overflow in <code>BytesMut::reserve</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/tokio-rs/bytes/commit/417dccdeff249e0c011327de7d92e0d6fbe7cc43"><code>417dccd</code></a>
Release bytes v1.11.1 (<a
href="https://redirect.github.com/tokio-rs/bytes/issues/820">#820</a>)</li>
<li><a
href="https://github.com/tokio-rs/bytes/commit/d0293b0e35838123c51ca5dfdf468ecafee4398f"><code>d0293b0</code></a>
Merge commit from fork</li>
<li>See full diff in <a
href="https://github.com/tokio-rs/bytes/compare/v1.11.0...v1.11.1">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/apache/iceberg-rust/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump time from 0.3.44 to 0.3.47 in /bindings/python (#2282)
Bumps [time](https://github.com/time-rs/time) from 0.3.44 to 0.3.47.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/time-rs/time/releases">time's
releases</a>.</em></p>
<blockquote>
<h2>v0.3.47</h2>
<p>See the <a
href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">changelog</a>
for details.</p>
<h2>v0.3.46</h2>
<p>See the <a
href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">changelog</a>
for details.</p>
<h2>v0.3.45</h2>
<p>See the <a
href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">changelog</a>
for details.</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">time's
changelog</a>.</em></p>
<blockquote>
<h2>0.3.47 [2026-02-05]</h2>
<h3>Security</h3>
<ul>
<li>
<p>The possibility of a stack exhaustion denial of service attack when
parsing RFC 2822 has been
eliminated. Previously, it was possible to craft input that would cause
unbounded recursion. Now,
the depth of the recursion is tracked, causing an error to be returned
if it exceeds a reasonable
limit.</p>
<p>This attack vector requires parsing user-provided input, with any
type, using the RFC 2822 format.</p>
</li>
</ul>
<h3>Compatibility</h3>
<ul>
<li>Attempting to format a value with a well-known format (i.e. RFC
3339, RFC 2822, or ISO 8601) will
error at compile time if the type being formatted does not provide
sufficient information. This
would previously fail at runtime. Similarly, attempting to format a
value with ISO 8601 that is
only configured for parsing (i.e. <code>Iso8601::PARSING</code>) will
error at compile time.</li>
</ul>
<h3>Added</h3>
<ul>
<li>Builder methods for format description modifiers, eliminating the
need for verbose initialization
when done manually.</li>
<li><code>date!(2026-W01-2)</code> is now supported. Previously, a space
was required between <code>W</code> and <code>01</code>.</li>
<li><code>[end]</code> now has a <code>trailing_input</code> modifier
which can either be <code>prohibit</code> (the default) or
<code>discard</code>. When it is <code>discard</code>, all remaining
input is ignored. Note that if there are components
after <code>[end]</code>, they will still attempt to be parsed, likely
resulting in an error.</li>
</ul>
<h3>Changed</h3>
<ul>
<li>More performance gains when parsing.</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>If manually formatting a value, the number of bytes written was one
short for some components.
This has been fixed such that the number of bytes written is always
correct.</li>
<li>The possibility of integer overflow when parsing an owned format
description has been effectively
eliminated. This would previously wrap when overflow checks were
disabled. Instead of storing the
depth as <code>u8</code>, it is stored as <code>u32</code>. This would
require multiple gigabytes of nested input to
overflow, at which point we've got other problems and trivial
mitigations are available by
downstream users.</li>
</ul>
<h2>0.3.46 [2026-01-23]</h2>
<h3>Added</h3>
<ul>
<li>All possible panics are now documented for the relevant
methods.</li>
<li>The need to use <code>#[serde(default)]</code> when using custom
<code>serde</code> formats is documented. This applies
only when deserializing an <code>Option<T></code>.</li>
<li><code>Duration::nanoseconds_i128</code> has been made public,
mirroring
<code>std::time::Duration::from_nanos_u128</code>.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/time-rs/time/commit/d5144cd2874862d46466c900910cd8577d066019"><code>d5144cd</code></a>
v0.3.47 release</li>
<li><a
href="https://github.com/time-rs/time/commit/f6206b050fd54817d8872834b4d61f605570e89b"><code>f6206b0</code></a>
Guard against integer overflow in release mode</li>
<li><a
href="https://github.com/time-rs/time/commit/1c63dc7985b8fa26bd8c689423cc56b7a03841ee"><code>1c63dc7</code></a>
Avoid denial of service when parsing Rfc2822</li>
<li><a
href="https://github.com/time-rs/time/commit/5940df6e72efb63d246ca1ca59a0f836ad32ad8a"><code>5940df6</code></a>
Add builder methods to avoid verbose construction</li>
<li><a
href="https://github.com/time-rs/time/commit/00881a4da1bc5a6cb6313052e5017dbd7daa40f0"><code>00881a4</code></a>
Manually format macros everywhere</li>
<li><a
href="https://github.com/time-rs/time/commit/bb723b6d826e46c174d75cd08987061984b0ceb7"><code>bb723b6</code></a>
Add <code>trailing_input</code> modifier to <code>end</code></li>
<li><a
href="https://github.com/time-rs/time/commit/31c4f8e0b56e6ae24fe0d6ef0e492b6741dda783"><code>31c4f8e</code></a>
Permit <code>W12</code> in <code>date!</code> macro</li>
<li><a
href="https://github.com/time-rs/time/commit/490a17bf306576850f33a86d3ca95d96db7b1dcd"><code>490a17b</code></a>
Mark error paths in well-known formats as cold</li>
<li><a
href="https://github.com/time-rs/time/commit/6cb1896a600be1538ecfab8f233fe9cfe9fa8951"><code>6cb1896</code></a>
Optimize <code>Rfc2822</code> parsing</li>
<li><a
href="https://github.com/time-rs/time/commit/6d264d59c25e3da0453c3defebf4640b0086a006"><code>6d264d5</code></a>
Remove erroneous <code>#[inline(never)]</code> attributes</li>
<li>Additional commits viewable in <a
href="https://github.com/time-rs/time/compare/v0.3.44...v0.3.47">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/apache/iceberg-rust/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* docs: Add DataFusion Comet to Users section of README (#2283)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- N/A.
## What changes are included in this PR?
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
- Add DataFusion Comet to the list of users with a description.
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
- N/A.
* feat(encryption) [1/N] Support encryption: Add crypto for AES-GCM (#2026)
Add Core Encryption Primitives for Iceberg Encryption Support.
Part of https://github.com/apache/iceberg-rust/issues/2034
## Summary
This PR introduces the foundational cryptographic primitives needed for
implementing encryption in iceberg-rust, providing AES-GCM encryption
operations that match the Java implementation's behavior and data
format.
## Motivation
Iceberg's Java implementation supports table-level encryption to protect
sensitive data at rest. To achieve feature parity and ensure
interoperability between Java and Rust implementations, we need to build
encryption support from the ground up. This PR provides the core
cryptographic operations that will serve as the foundation for the
complete encryption feature.
## Changes
New Module: encryption
Added a new encryption module with core AES-GCM cryptographic
operations:
- encryption/crypto.rs - Core encryption implementation
- EncryptionAlgorithm enum supporting AES-128-GCM as this is the only
algorithm currently supported in arrow parquet
- SecureKey struct with automatic memory zeroization for security
- AesGcmEncryptor providing encrypt/decrypt operations with AAD support
Key Features
1. Java-Compatible Format: Ciphertext format matches Java's
implementation exactly:
[12-byte nonce][encrypted data][16-byte GCM authentication tag]
1. This ensures files encrypted by Java can be decrypted by Rust and
vice versa.
2. Secure Key Handling: Uses the zeroize crate to automatically clear
encryption keys from memory when dropped, preventing key material from
lingering in memory.
3. Additional Authenticated Data (AAD): Full support for AAD to ensure
integrity of associated metadata that isn't encrypted.
4. Comprehensive Testing: 8 tests covering:
- Round-trip encryption/decryption for both AES-128 and AES-256
- AAD validation
- Empty plaintext handling
- Tamper detection
- Format compatibility verification
Dependencies Added
- aes-gcm = "0.10" - Industry-standard AES-GCM implementation
- zeroize = "1.7" - Secure memory cleanup for encryption keys
Compatibility
This implementation directly corresponds to Java's
https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/encryption/Ciphers.java:
| Java Class | Rust Implementation |
|-----------------------------|------------------------------------------|
| Ciphers.AesGcmEncryptor | AesGcmEncryptor::encrypt() |
| Ciphers.AesGcmDecryptor | AesGcmEncryptor::decrypt() |
| EncryptionAlgorithm.AES_GCM | EncryptionAlgorithm::Aes128Gcm|
Testing
Future Work
This PR is the first in a series to implement full encryption support.
Upcoming PRs will add:
1. Table properties for encryption configuration
2. Key management interfaces (KeyManagementClient trait)
3. EncryptionManager implementation
4. Native Parquet encryption integration
5. AWS KMS support
6. Integration with Table and FileIO
Review Notes
- This PR is intentionally minimal and self-contained
- No existing code paths are modified - this is purely additive
- The module is public but won't be used until future PRs wire it up
- Format compatibility with Java has been prioritized to ensure
interoperability
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #. https://github.com/apache/iceberg-rust/issues/2035
## What changes are included in this PR?
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
Yes
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* feat!(catalog): adding support for purge_table (#2232)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #2133
## What changes are included in this PR?
- Add catalog/utils.rs to provide helpers to delete table data using
file_io and table_metadata
- Add new API `purge_table` to `Catalog` trait and add default
implementation
- Implement purge_table for S3TableCatalog and RestCatalog
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
Added new tests in table_suite
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* deps: upgrade DataFusion to 53.0, Arrow to 58 (#2206)
## Which issue does this PR close?
- Closes #.
## What changes are included in this PR?
- Bump DataFusion to 53.0.0, Arrow/Parquet to 58, sqllogictest to 0.29,
pyo3 to 0.28.
- Adapt to DataFusion 53 API changes in physical plan executors and
python bindings.
- Update SLT expected test output.
## Are these changes tested?
Existing tests.
---------
Co-authored-by: Xander <zander181@googlemail.com>
* chore(deps): Bump uuid from 1.22.0 to 1.23.0 (#2291)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.22.0 to 1.23.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/uuid-rs/uuid/releases">uuid's
releases</a>.</em></p>
<blockquote>
<h2>v1.23.0</h2>
<h2>What's Changed</h2>
<ul>
<li>feat: add support for 'hyphenated' format in the serde module by <a
href="https://github.com/FrenchDilettante"><code>@FrenchDilettante</code></a>
in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/865">uuid-rs/uuid#865</a></li>
<li>Fix a number of bugs in time-related code by <a
href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/872">uuid-rs/uuid#872</a></li>
<li>Reword invalid char error message by <a
href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/873">uuid-rs/uuid#873</a></li>
<li>Impl cleanups by <a
href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/874">uuid-rs/uuid#874</a></li>
<li>Use LazyLock to synchronize v1/v6 context initialization by <a
href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/875">uuid-rs/uuid#875</a></li>
<li>Prepare for 1.23.0 release by <a
href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/876">uuid-rs/uuid#876</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/FrenchDilettante"><code>@FrenchDilettante</code></a>
made their first contribution in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/865">uuid-rs/uuid#865</a></li>
</ul>
<h2>Special thanks</h2>
<p><a href="https://github.com/meng-xu-cs"><code>@meng-xu-cs</code></a>
raised a series of bugs against the timestamp logic in <code>uuid</code>
using automated tooling. The issues themselves were reasonably and
responsibly presented and the end result is a better <code>uuid</code>
library for everyone. Thanks!</p>
<h1>Deprecations</h1>
<p>This release includes the following deprecations:</p>
<ul>
<li><code>Context</code>: Renamed to <code>ContextV1</code></li>
<li><code>Timestamp::from_gregorian</code>: Renamed to
<code>Timestamp::from_gregorian_time</code></li>
</ul>
<h1>Change to <code>Version::Max</code></h1>
<p><code>Version::Max</code>'s <code>u8</code> representation has
changed from <code>0xff</code> to <code>0x0f</code> to match the value
returned by <code>Uuid::get_version_num</code>.</p>
<h1>Change to <code>Uuid::get_version</code> for the max UUID</h1>
<p><code>Uuid::get_version</code> will only return
<code>Some(Version::Max)</code> if the UUID is actually the max UUID
(all bytes are <code>0xff</code>). Previously it would return
<code>Some</code> if only the version field was <code>0x0f</code>. This
change matches the behaviour of the nil UUID, which only returns
<code>Some(Version::Nil)</code> if the UUID is the nil UUID (all bytes
are <code>0x00</code>).</p>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/uuid-rs/uuid/compare/v1.22.0...v1.23.0">https://github.com/uuid-rs/uuid/compare/v1.22.0...v1.23.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/uuid-rs/uuid/commit/00ab922d5351607dfff520f37eb49cb9854fda73"><code>00ab922</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/876">#876</a> from
uuid-rs/cargo/v1.23.0</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/726ba45fe3491bf6253173d0be6b99ed3b1cbbb9"><code>726ba45</code></a>
prepare for 1.23.0 release</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/996dadea029e3976f52cba58e5e9b9a08c4f82c4"><code>996dade</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/875">#875</a> from
uuid-rs/fix/context-ordering</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/e14047993bc5a6180a96119436a983c19d79b084"><code>e140479</code></a>
simplify a use stmt</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/8ed9142847a22bc7707794bfee6b2016d4470772"><code>8ed9142</code></a>
reorganize and document more v7 context internals</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/e09a3225a8d99c5eadcbbeb7432195b2ea5ece76"><code>e09a322</code></a>
use LazyLock to synchronize v1/v6 context initialization</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/0f260cc67135ac20d914e387a47e59960247fdee"><code>0f260cc</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/874">#874</a> from
uuid-rs/chore/impl-cleanups</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/1419e91097fcffc7afa8f54eb41fdc912200b540"><code>1419e91</code></a>
clean up and refactor main lib tests</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/ceeaf4b7b59895497c59acdaf286233b1e7cc576"><code>ceeaf4b</code></a>
ensure we don't overflow on counters less than 12</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/63bc8f52e5042b9c729fa0380b9948b49fe397cc"><code>63bc8f5</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/873">#873</a> from
uuid-rs/fix/error-msg</li>
<li>Additional commits viewable in <a
href="https://github.com/uuid-rs/uuid/compare/v1.22.0...v1.23.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* ci: improve github workflows (#2289)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #.
## What changes are included in this PR?
Relates to https://github.com/apache/iceberg/issues/15742
This PR
- Add "ASF allowlist check"
- Pin commit for codeql.yml (zizmor recommended)
- Add back Github Action auto-update for dependabot (reverts #2267)
- Add cooldown to dependabot (zizmor recommended)
- `Swatinem/rust-cache@v2` -> `swatinem/rust-cache@v2` (fix case
sensitivity) [asf infra allowlist uses
lowercase](https://github.com/apache/infrastructure-actions/blob/fae466bc0d9821859a623cbc7648c750ff359ec6/approved_patterns.yml#L271)
We can add back dependabot for github action because the "ASF allowlist
check" will now alert when an action is not allowed (failures will no
longer be silent)
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* Make `convert_filters_to_predicate` public (#2118)
## What changes are included in this PR?
- Make `convert_filters_to_predicate` public in the DataFusion
integration to allow external usage of the filter conversion logic.
## Are these changes tested?
- This is a visibility change (`pub use`) and does not introduce new
logic.
Co-authored-by: Denis Semenov <d.s.semenov@vkteam.ru>
* ci: fix zizmor security findings (#2290)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
Relates to https://github.com/apache/iceberg/issues/15742
Follow up to #2289
## What changes are included in this PR?
Fix github workflow based on zizmor recommendation for security best
practice
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
Yes
```
➜ iceberg-rust git:(kevinjqliu/zizmor-fix) uvx --from zizmor zizmor --offline .github/
🌈 zizmor v1.23.1
INFO audit: zizmor: 🌈 completed .github/actions/get-msrv/action.yml
INFO audit: zizmor: 🌈 completed .github/actions/overwrite-package-version/action.yml
INFO audit: zizmor: 🌈 completed .github/actions/setup-builder/action.yml
INFO audit: zizmor: 🌈 completed .github/dependabot.yml
INFO audit: zizmor: 🌈 completed .github/workflows/audit.yml
INFO audit: zizmor: 🌈 completed .github/workflows/bindings_python_ci.yml
INFO audit: zizmor: 🌈 completed .github/workflows/ci.yml
INFO audit: zizmor: 🌈 completed .github/workflows/ci_typos.yml
INFO audit: zizmor: 🌈 completed .github/workflows/codeql.yml
INFO audit: zizmor: 🌈 completed .github/workflows/publish.yml
INFO audit: zizmor: 🌈 completed .github/workflows/release_python.yml
INFO audit: zizmor: 🌈 completed .github/workflows/release_python_nightly.yml
INFO audit: zizmor: 🌈 completed .github/workflows/stale.yml
INFO audit: zizmor: 🌈 completed .github/workflows/website.yml
No findings to report. Good job! (1 ignored, 37 suppressed)
```
* fix(spec): clean up -1 snapshot ID sentinel usage and add deserialization test (#2294)
## Which issue does this PR close?
- Closes #352.
## What changes are included in this PR?
- Replaces hardcoded `-1` with `EMPTY_SNAPSHOT_ID` constant in table
metadata deserialization.
- Adds `test_empty_snapshot_id_is_normalized_to_none` to verify that the
Java-style `-1` sentinel for `current-snapshot-id` is normalized to
`None` during deserialization.
- Removes the public `UNASSIGNED_SNAPSHOT_ID` constant and moving it to
a private constant scoped to the manifest writer module.
## Are these changes tested?
Adds a test `test_empty_snapshot_id_is_normalized_to_none` verifying the
deserialization normalization.
* feat!: Enhance compression codec enum. (#2288)
## Which issue does this PR close?
This is an intermediate PR for #1731
I'm splitting out changes from
https://github.com/apache/iceberg-rust/pull/1851 to the compression
codec to make it easier to review. Once we decide on approach here and
merge it I'll update https://github.com/apache/iceberg-rust/pull/1851
accordingly.
## What changes are included in this PR?
- Add optional compression level to gzip and zstd (needed for when avro
compression usage).
- Add Snappy as a compression codec (also will be used for Avro)
- Manually code up some previously auto-generated methods as a result.
AI helped with an initial version of this PR.
## Are these changes tested?
Additional unit tests
* ci: allows nightly to run on workflow dispatch (#2304)
* ci: ensure use bash shell for env (#2305)
* feat(iceberg): Add snapshot utils to scan ancestors (#2285)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #2241
## What changes are included in this PR?
- Add `Ancestors` to help scan past snapshots
- Moved existing util to the new utils mod
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
Yes
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
* Fix zizmor workflow (#2324)
## Which issue does this PR close?
- Closes #2323 .
## What changes are included in this PR?
## Are these changes tested?
ci
* chore(deps): Bump crate-ci/typos from 1.44.0 to 1.45.0 (#2318)
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.44.0 to
1.45.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/crate-ci/typos/releases">crate-ci/typos's
releases</a>.</em></p>
<blockquote>
<h2>v1.45.0</h2>
<h2>[1.45.0] - 2026-04-01</h2>
<h3>Features</h3>
<ul>
<li>Updated the dictionary with the <a
href="https://redirect.github.com/crate-ci/typos/issues/1509">March
2026</a> changes</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/crate-ci/typos/blob/master/CHANGELOG.md">crate-ci/typos's
changelog</a>.</em></p>
<blockquote>
<h1>Change Log</h1>
<p>All notable changes to this project will be documented in this
file.</p>
<p>The format is based on <a href="https://keepachangelog.com/">Keep a
Changelog</a>
and this project adheres to <a href="https://semver.org/">Semantic
Versioning</a>.</p>
<!-- raw HTML omitted -->
<h2>[Unreleased] - ReleaseDate</h2>
<h2>[1.45.0] - 2026-04-01</h2>
<h3>Features</h3>
<ul>
<li>Updated the dictionary with the <a
href="https://redirect.github.com/crate-ci/typos/issues/1509">March
2026</a> changes</li>
</ul>
<h2>[1.44.0] - 2026-02-27</h2>
<h3>Features</h3>
<ul>
<li>Updated the dictionary with the <a
href="https://redirect.github.com/crate-ci/typos/issues/1488">February
2026</a> changes</li>
</ul>
<h2>[1.43.5] - 2026-02-16</h2>
<h3>Fixes</h3>
<ul>
<li><em>(pypi)</em> Hopefully fix the sdist build</li>
</ul>
<h2>[1.43.4] - 2026-02-09</h2>
<h3>Fixes</h3>
<ul>
<li>Don't correct <code>pincher</code></li>
</ul>
<h2>[1.43.3] - 2026-02-06</h2>
<h3>Fixes</h3>
<ul>
<li><em>(action)</em> Adjust how typos are reported to github</li>
</ul>
<h2>[1.43.2] - 2026-02-05</h2>
<h3>Fixes</h3>
<ul>
<li>Don't correct <code>certifi</code> in Python</li>
</ul>
<h2>[1.43.1] - 2026-02-03</h2>
<h3>Fixes</h3>
<ul>
<li>Don't correct <code>consts</code></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/crate-ci/typos/commit/02ea592e44b3a53c302f697cddca7641cd051c3d"><code>02ea592</code></a>
chore: Release</li>
<li><a
href="https://github.com/crate-ci/typos/commit/b859c0df7f391deba73030f79b957e62b4d81dc6"><code>b859c0d</code></a>
chore: Release</li>
<li><a
href="https://github.com/crate-ci/typos/commit/6fd32cee0c74337fd419788965d8f1c96d6f87a5"><code>6fd32ce</code></a>
docs: Update changelog</li>
<li><a
href="https://github.com/crate-ci/typos/commit/7626d89232db5fc0c23e290107cf420047e297e9"><code>7626d89</code></a>
Merge pull request <a
href="https://redirect.github.com/crate-ci/typos/issues/1530">#1530</a>
from crate-ci/renovate/j178-prek-action-2.x</li>
<li><a
href="https://github.com/crate-ci/typos/commit/2c9510cd3b9756411e186a1ff1d0ee9a0bb61895"><code>2c9510c</code></a>
Merge pull request <a
href="https://redirect.github.com/crate-ci/typos/issues/1532">#1532</a>
from epage/march</li>
<li><a
href="https://github.com/crate-ci/typos/commit/265b88f6490711bebac4bb109149f1f06c2255e9"><code>265b88f</code></a>
feat(dict): March updates</li>
<li><a
href="https://github.com/crate-ci/typos/commit/5baf2ce236258ea037d71f364aa7a13eb1d2cf70"><code>5baf2ce</code></a>
chore(deps): Update compatible (<a
href="https://redirect.github.com/crate-ci/typos/issues/1529">#1529</a>)</li>
<li><a
href="https://github.com/crate-ci/typos/commit/0442cb7c40aa99aa5a9ec99ac4c3f0c654260f34"><code>0442cb7</code></a>
chore(deps): Update j178/prek-action action to v2</li>
<li><a
href="https://github.com/crate-ci/typos/commit/8f11c0dc0f31c780c45b3dd5b72ff4b48a350b75"><code>8f11c0d</code></a>
Merge pull request <a
href="https://redirect.github.com/crate-ci/typos/issues/1524">#1524</a>
from epage/update</li>
<li><a
href="https://github.com/crate-ci/typos/commit/ecdbfab467d6037f796c649e7616aee4a6a16c5a"><code>ecdbfab</code></a>
chore: Update dependencies</li>
<li>Additional commits viewable in <a
href="https://github.com/crate-ci/typos/compare/631208b7aac2daa8b707f55e7331f9112b0e062d...02ea592e44b3a53c302f697cddca7641cd051c3d">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: blackmwk <liurenjie1024@outlook.com>
* chore(deps): Bump taiki-e/install-action from 2.70.0 to 2.73.0 (#2319)
Bumps
[taiki-e/install-action](https://github.com/taiki-e/install-action) from
2.70.0 to 2.73.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/taiki-e/install-action/releases">taiki-e/install-action's
releases</a>.</em></p>
<blockquote>
<h2>2.73.0</h2>
<ul>
<li>
<p>Introduce <a
href="https://blog.yossarian.net/2025/11/21/We-should-all-be-using-dependency-cooldowns">dependency
cooldown</a> when installing with
<code>taiki-e/install-action@<tool_name></code>, <code>tool:
<tool_name>@latest</code>, or <code>tool:
<tool_name>@<omitted_version></code> to mitigate the risk of
supply chain attacks by default. (<a
href="https://redirect.github.com/taiki-e/install-action/pull/1666">#1666</a>)</p>
<p>This action without this cooldown already takes a few hours to a few
days for new releases to be reflected (as with other common package
managers that verify checksums or signatures), so this should not affect
most users.</p>
<p>See the <a
href="https://github.com/taiki-e/install-action#security">"Security"
section in readme</a> for more details.</p>
</li>
<li>
<p>Improve robustness for network failure.</p>
</li>
<li>
<p>Documentation improvements.</p>
</li>
</ul>
<h2>2.72.0</h2>
<ul>
<li>
<p>Support <code>cargo-xwin</code>. (<a
href="https://redirect.github.com/taiki-e/install-action/pull/1659">#1659</a>,
thanks <a
href="https://github.com/daxpedda"><code>@daxpedda</code></a>)</p>
</li>
<li>
<p>Support trailing comma in <code>tool</code> input option.</p>
</li>
<li>
<p>Update <code>tombi@latest</code> to 0.9.14.</p>
</li>
</ul>
<h2>2.71.3</h2>
<ul>
<li>
<p>Update <code>wasm-tools@latest</code> to 1.246.2.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2026.4.3.</p>
</li>
</ul>
<h2>2.71.2</h2>
<ul>
<li>
<p>Implement workaround for <a
href="https://redirect.github.com/actions/partner-runner-images/issues/169">windows-11-arm
runner bug</a> which sometimes causes installation failure. (<a
href="https://redirect.github.com/taiki-e/install-action/pull/1657">#1657</a>)</p>
<p>This addresses an issue that was attempted to be worked around in
2.71.0 but was insufficient.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2026.4.1.</p>
</li>
<li>
<p>Update <code>uv@latest</code> to 0.11.3.</p>
</li>
</ul>
<h2>2.71.1</h2>
<ul>
<li>
<p>Fix a regression that caused an execution policy violation on
self-hosted Windows runner due to use of non-default
<code>powershell</code> shell, introduced in 2.71.0.</p>
</li>
<li>
<p>Update <code>dprint@latest</code> to 0.53.2.</p>
</li>
</ul>
<h2>2.71.0</h2>
<ul>
<li>
<p>Support <code>wasm-tools</code>. (<a
href="https://redirect.github.com/taiki-e/install-action/pull/1642">#1642</a>,
thanks <a
href="https://github.com/crepererum"><code>@crepererum</code></a>)</p>
</li>
<li>
<p>Support <code>covgate</code>. (<a
href="https://redirect.github.com/taiki-e/install-action/pull/1613">#1613</a>,
thanks <a
href="https://github.com/jesse-black"><code>@jesse-black</code></a>)</p>
</li>
<li>
<p>Implement potential workaround for <a
href="https://redirect.github.com/actions/partner-runner-images/issues/169">windows-11-arm
runner bug</a> which sometimes causes issue that the action successfully
completes but the tool is not installed. (<a
href="https://redirect.github.com/taiki-e/install-action/pull/1647">#1647</a>)</p>
</li>
<li>
<p>Update <code>typos@latest</code> to 1.45.0.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2026.4.0.</p>
</li>
<li>
<p>Update <code>cargo-careful@latest</code> to 0.4.10.</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md">taiki-e/install-action's
changelog</a>.</em></p>
<blockquote>
<h1>Changelog</h1>
<p>All notable changes to this project will be documented in this
file.</p>
<p>This project adheres to <a href="https://semver.org">Semantic
Versioning</a>.</p>
<!-- raw HTML omitted -->
<h2>[Unreleased]</h2>
<ul>
<li>
<p>Update <code>vacuum@latest</code> to 0.25.6.</p>
</li>
<li>
<p>Update <code>gungraun-runner@latest</code> to 0.18.1.</p>
</li>
</ul>
<h2>[2.75.7] - 2026-04-11</h2>
<ul>
<li>
<p>Update <code>covgate@latest</code> to 0.1.4.</p>
</li>
<li>
<p>Update <code>wasm-bindgen@latest</code> to 0.2.118.</p>
</li>
</ul>
<h2>[2.75.6] - 2026-04-11</h2>
<ul>
<li>
<p>Update <code>mise@latest</code> to 2026.4.8.</p>
</li>
<li>
<p>Update <code>cargo-deny@latest</code> to 0.19.1.</p>
</li>
</ul>
<h2>[2.75.5] - 2026-04-10</h2>
<ul>
<li>
<p>Update <code>biome@latest</code> to 2.4.11.</p>
</li>
<li>
<p>Update <code>wasmtime@latest</code> to 43.0.1.</p>
</li>
<li>
<p>Update <code>uv@latest</code> to 0.11.6.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2026.4.7.</p>
</li>
<li>
<p>Update <code>gungraun-runner@latest</code> to 0.18.0.</p>
</li>
</ul>
<h2>[2.75.4] - 2026-04-10</h2>
<ul>
<li>
<p>Enhance security when cargo-binstall fallback is enabled. (<a
href="https://github.com/taiki-e/install-action/commit/08a38582e35739e1fda5e4cd298e348536a1ddaa">08a38582</a>,
<a
href="https://github.com/taiki-e/install-action/commit/ba626b4d71edcae17f6ca956b7b81e925b2d24ef">ba626b4d</a>)</p>
</li>
<li>
<p>Update <code>martin@latest</code> to 1.5.0.</p>
</li>
<li>
<p>Update <code>uv@latest</code> to 0.11.5.</p>
</li>
<li>
<p>Update <code>syft@latest</code> to 1.42.4.</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/taiki-e/install-action/commit/0abfcd587b70a713fdaa7fb502c885e2112acb15"><code>0abfcd5</code></a>
Release 2.75.7</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/cf81fe383b77d63972a04bb1e84d163e13d3a584"><code>cf81fe3</code></a>
Update <code>covgate@latest</code> to 0.1.4</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/e8aec859b7eb3ba8572ade244032d91139e4b75f"><code>e8aec85</code></a>
Update <code>wasm-bindgen@latest</code> to 0.2.118</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/d053c7f943c3a577e98f342e9e28edf09931644c"><code>d053c7f</code></a>
Release 2.75.6</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/9019f2ebb5fcc4be56ccca93d779a8a14f83bb42"><code>9019f2e</code></a>
Update <code>mise@latest</code> to 2026.4.8</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/1b07fb74591cf228b32613bd9ca519dc027ae914"><code>1b07fb7</code></a>
Update <code>cargo-deny@latest</code> to 0.19.1</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/be1e8a863739de1370a81e1c5ad4babbab50c914"><code>be1e8a8</code></a>
Update vacuum manifest</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/1cc7501bf80d38541e25e6a90f1f1c10054297c9"><code>1cc7501</code></a>
Update gungraun-runner manifest</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/44afe623bee1f92436bd2e2d30d56626792cd0f0"><code>44afe62</code></a>
Update covgate manifest</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/7a4939c09608b2a1986b484eca1d16fd0db8ebef"><code>7a4939c</code></a>
Release 2.75.5</li>
<li>Additional commits viewable in <a
href="https://github.com/taiki-e/install-action/compare/0fde6d128a3d980ceac30be8c8b8739abd963b81...0abfcd587b70a713fdaa7fb502c885e2112acb15">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: blackmwk <liurenjie1024@outlook.com>
* chore(deps): Bump aws-sdk-s3tables from 1.53.0 to 1.54.0 (#2320)
Bumps [aws-sdk-s3tables](https://github.com/awslabs/aws-sdk-rust) from
1.53.0 to 1.54.0.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/awslabs/aws-sdk-rust/commits">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: blackmwk <liurenjie1024@outlook.com>
* Fix ci workflow failure (#2325)
## Which issue does this PR close?
- Closes #.
## What changes are included in this PR?
Our current ci failed due to an audit issue, see:
https://github.com/apache/iceberg-rust/pull/2321 . This pr fix the ci
failure.
## Are these changes tested?
CI.
* chore(deps): Bump tokio from 1.50.0 to 1.51.0 (#2321)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.50.0 to 1.51.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/tokio-rs/tokio/releases">tokio's
releases</a>.</em></p>
<blockquote>
<h2>Tokio v1.51.0</h2>
<h1>1.51.0 (April 3rd, 2026)</h1>
<h3>Added</h3>
<ul>
<li>net: implement <code>get_peer_cred</code> on Hurd (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7989">#7989</a>)</li>
<li>runtime: add <code>tokio::runtime::worker_index()</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7921">#7921</a>)</li>
<li>runtime: add runtime name (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7924">#7924</a>)</li>
<li>runtime: stabilize <code>LocalRuntime</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7557">#7557</a>)</li>
<li>wasm: add wasm32-wasip2 networking support (<a
href="https://redirect.github.com/tokio-rs/tokio/…
Previously these properties where not honored on tabel properties. - Adds table properties for these values. - Plumbs them through for writers.
ef52dde to
c0ca9c3
Compare
Which issue does this PR close?
What changes are included in this PR?
Previously these properties where not honored on table properties.
AI was used to help generate tests and some of the plumbing.
Are these changes tested?
Added unit tests
BREAKING CHANGE: Adds codec parameter to some public functions. By default start compressing manifests and manifest lists.