You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Move iceberg-rust 0.10.1 to DataFusion 55 and Arrow 59 - #50
The Tests job could not start the integration fixtures on this line: make docker-up failed before any test ran. #51 cherry-picks two upstream CI fixes. Both are test-infrastructure changes, independent of the DataFusion 55 / Arrow 59 move:
DataFusion 54 -> 55.1.0, arrow/parquet 58 -> 59. pyo3 0.28 -> 0.29, the version
arrow 59's pyarrow support links (the python bindings otherwise pull a second
libpython link). serde_arrow's dev-dependency feature arrow-58 -> arrow-59, so
the tests build one arrow.
- TableReference is imported from datafusion::common.
- CreateExternalTable.location became locations; an iceberg external table
takes exactly one metadata file and returns an error otherwise.
- ExecutionPlan::apply_expressions: none of the scan/write/commit/metadata nodes
holds a PhysicalExpr (filters are iceberg predicates), so they return Continue.
Spice links iceberg-rust against spiceai/arrow-rs spiceai-59-patches and
spiceai/datafusion spiceai-55-patches, while this workspace resolved stock
crates.io arrow 59.3.0 and DataFusion 55.1.0, so its tests never exercised the
code Spice ships. Patch every arrow/parquet crate (db730b42) and DataFusion
crate (75759ff7) in the graph to those forks, and regenerate Cargo.lock.
…le::new_with_value
pyo3 0.29 deprecates PyCapsule::new, which fails `make check-clippy` under
-D warnings. new_with_value takes the name as a &'static CStr, so the capsule
name becomes a c-string literal and the CString allocation (and its unwrap)
goes away. Both store the value at the start of the capsule's allocation and
drop it when the capsule is destroyed.
`make check-public-api` failed for iceberg and iceberg-datafusion: the Spice
additions (FileScanTask::limit, TableScanBuilder::with_limit,
TableScan::to_arrow_from_tasks, Transaction::row_delta, the snapshot-pinned
IcebergTableProvider and IcebergTableScan constructors) were never recorded,
and DataFusion 55 moves the TableProvider, TableProviderFactory and
CatalogProvider traits these types implement to datafusion_catalog.
Regenerated with `make generate-public-api`.
CI note: Tests fail because the MinIO images are gone from Docker Hub and the HMS image build hits a 404 from debian-security; #51 backports upstream apache#3273 (RustFS) and apache#3174, and Tests pass there (1859 passed). asf-allowlist-check and zizmor fail identically on spiceai-0.10.1-df-54 #49 (upstream's own workflow pins).
… HMS jar ADD) (#51)
* ci: replace MinIO with RustFS for integration tests
Backport of apache#3273 (119fad9). Docker Hub no longer serves
minio/mc or minio/minio, and quay.io/minio no longer allows anonymous pulls, so
`make docker-up` fails before any integration test runs. Conflict in
dev/docker-compose.yaml resolved to the RustFS services, keeping this line's
REST fixture CATALOG_URI.
(cherry picked from commit 119fad9)
* fix: Fix CI failure due to integration test's Hive install (apache#3174)
Install Hive directly
Signed-off-by: Jannik Steinmann <jannik.steinmann@datadoghq.com>
(cherry picked from commit 4687d26)
---------
Signed-off-by: Jannik Steinmann <jannik.steinmann@datadoghq.com>
Co-authored-by: Jannik Steinmann <jannik.steinmann@datadoghq.com>
Regenerate dependency inventories for the DataFusion and Arrow upgrade
Cargo.toml:83
Regenerate every package's DEPENDENCIES.rust.tsv for this upgrade. The checked-in inventories still list DataFusion 53 and Arrow 58 (for example, crates/integrations/datafusion/DEPENDENCIES.rust.tsv:18-30,88-118), while dev/release/dependencies.sh:23 and website/src/release.md:152-163 establish that these are version-sensitive generated license lists. As committed, the dependency/license metadata does not describe the resolved graph.
That review's three threads were resolved earlier without a reply. Its headline: location validation is untested, the RustFS scope is undocumented, and the fork branch notes are stale. This PR's head branch only takes changes through a PR (the spiceai-* ruleset declined a direct push), so the code fixes are in #54, which targets this PR's head.
Fork branch notes (thread): Copilot is right on both lines; fixed in test(datafusion): cover the factory's one-location check; fix fork branch notes #54. From the GitHub compare API: spiceai/arrow-rs spiceai-59...2e2cc330 is identical, while spiceai-59-patches...2e2cc330 is diverged (6 ahead, 3 behind). spiceai/datafusion spiceai-55...02550cf9 is identical, while spiceai-55-patches...02550cf9 is diverged (13 ahead, 27 behind). The notes on Cargo.toml lines 169 and 188 now name spiceai-59 and spiceai-55.
#54 has Copilot's 🟢 Approved on 10f62dd (review). It lands on this PR once it is merged into spiceai-0.10.1-df-55-patches.
Two checks fail on a0bd6b0 for reasons unrelated to this PR, and neither is required. This PR changes nothing under .github/ (git diff --stat origin/spiceai-0.10.1-df-55 HEAD -- .github/ prints nothing), and both checks fail the same way on #49, which targets the DataFusion 54 line:
Addressed #50 (comment): the CI note checks out against the runs. Before #51, Tests stopped at make docker-up with pull access denied for minio/mc (run 36782663275). With only the RustFS swap, the HMS image build got 404 Not Found from debian-security (run 36816859585). With both backports, CI passed (run 36817850351). The zizmor and allowlist failures on #49 are quoted above.
…anch notes (#54)
The DataFusion 55 port rejects CREATE EXTERNAL TABLE commands that carry
zero or several locations, but only the one-location path was tested. Add
a test for each case.
The arrow-rs and DataFusion pins are the heads of spiceai-59 and
spiceai-55, not of the -patches branches the comments named.
… spicebench at their merges
The fork PRs these pins pointed at have merged, mostly as squash merges, so
several pinned commits are no longer on their base branches. Each pin now
points at the merge on the fork's base branch:
- iceberg-rust spiceai-0.10.1-df-55 @ 3e2a14e (spiceai/iceberg-rust#50, #54)
- snowflake-rs spiceai-59 @ f555738 (spiceai/snowflake-rs#13, #15)
- spark-connect-rs spiceai-59-2 @ 18ae9bd (spiceai/spark-connect-rs#15, #16)
- spice-rs trunk @ 4429f39 (spiceai/spice-rs#103)
- spicebench trunk @ 8a90555 (spiceai/spicebench#284)
datafusion-federation stays at 9ca84a3, which is on spiceai-55 because
spiceai/datafusion-federation#88 was a merge commit; only its branch note
changes.
What changes for Spice: lint-only rewrites in snowflake-api, and tests in
iceberg-datafusion and spark-connect-core. The other differences are in
the forks' own [patch.crates-io] sections, which cargo ignores for a
dependency. spice-rs's tree is identical.
anush008
pushed a commit
to anush008/spiceai
that referenced
this pull request
Oct 6, 2026
* build: pin the DataFusion 55 / Arrow 59 fork lines
Moves every fork to its DataFusion 55 line: arrow-rs 59.3.0, DataFusion
55.1.0, Ballista (upstream main on DataFusion 55), federation,
table-providers, iceberg-rust 0.10.1, duckdb-rs 1.4.4 on arrow 59, ADBC 24,
snowflake-rs, spark-connect-rs, spice-rs and spicebench on arrow 59.
datafusion-functions-json comes from crates.io (0.55.6); delta-kernel stays
on 0.27.1 with its default engine split into delta_kernel_default_engine.
docs/dev/fork_patches.md is re-audited against the new lines; the pins stay
marked TEMPORARY until the fork pull requests merge.
* feat: upgrade to DataFusion 55 and Arrow 59
- Statistics flow through DataFusion 55's StatisticsContext. The deprecated
partition_statistics returns Absent for built-in plans on 55, so every
wrapper implements child_stats_requests / statistics_from_inputs and the
optimizer rules, join sizing and Flight batch sizing read real statistics
again.
- ExecutionPlan / TableProvider wrappers implement the new required and
defaulted methods explicitly (apply_expressions, replace_children,
try_to_proto, merge_into, ...), forwarding where they wrap.
- Arrow 59 refuses a MAP whose entries field is nullable: Flight SQL reads
replace the declared schema before decoding and normalize the map
afterwards, and legacy IPC maps decode as a list (arrow_tools map_entries).
- Delta Lake refuses Void and interval columns with a structured error
instead of mis-reading them.
- Ballista 55: Vcores, the protocol-version handshake, task-id based
cancellation and the append-only task-info model.
- Vortex 0.86 and the remaining API moves (TableReference, TableScanBuilder,
TableSchema builder, codec proto converter, WriteOp::MergeInto).
* ci: validate the datafusion-table-providers pin against spiceai-55-patches
The DataFusion 55 line of the fork is spiceai-55-patches.
* revert: drop the unfinished guard tests d65aa29 picked up by accident
d65aa29 was meant to change only the CI branch check; it also committed
work-in-progress test edits. Restore those files to 98800a8; the finished
tests land in their own commit.
* fix(duckdb): push regexp_count down NULL-preserving, as DataFusion 55 evaluates it
DataFusion 55 (apache/datafusion#24239) makes regexp_count propagate NULL,
as PostgreSQL does. The DuckDB rendering still wrapped the count in
coalesce(.., 0) to match the old kernel, so a DuckDB-accelerated dataset
answered 0 where local evaluation answers NULL, and a WHERE on the count kept
rows local evaluation drops. Render it as len(regexp_extract_all(..)), which
is NULL for a NULL input or pattern.
* test(cayenne): count -0.0 as equal to 0.0 in the boundary-values test
DataFusion 55 compares floats in SQL by IEEE 754 value, so float_val = 0.0
matches the -0.0 row too (4 rows, not 3). A MemTable in the same session
returns the same rows, so this is the expectation, not Cayenne.
* build: repin the forks and guard their new patches
- Ballista 8bf37ad0: the executor poll loop accepts a vcore semaphore with no
permits yet. Upstream's assert panicked the loop at startup, so no Spice
executor registered and every cluster query failed.
- DataFusion d98ae645 (MSRV-clean unparser), federation 9ca84a39,
table-providers 661d16d1, iceberg 575d81a1: their own tests now run
against the Spice DataFusion (and, for table-providers, ADBC) forks.
- Guard tests for the unparser rescope, eager aggregation through
StatisticsContext, SchemaCastScanExec statistics forwarding and Ballista
TaskInfo fields 11/12; the ledger names them and records the Ballista fix.
- The distributed EXPLAIN snapshot now reads Ballista 55's
UnknownPartitioning(1) where it printed None.
* build: point the fork pins at the spiceai-*-patches-2 branches
The DataFusion 55 lines of arrow-rs, datafusion, datafusion-ballista,
datafusion-federation, datafusion-table-providers, duckdb-rs, snowflake-rs and
spark-connect-rs are reviewed on new spiceai-*-patches-2 branches, leaving the
earlier spiceai-*-patches branches untouched. Same revisions; only the branch
names in the pin comments, the fork-patch ledger and the table-providers pin
check change.
* fix: re-root a delegated projection on the inner plan before swapping
DataFusion 55's projection pushdown reads a projection's input as the node
being swapped with: `ProjectionExec` collapses the chain starting at
`projection.input()`, `CoalescePartitionsExec` rebuilds from its child. A
wrapper that handed its inner plan the projection unchanged (its input being
the wrapper) got it back still above the wrapper, and the pushdown then nested
another copy on every step: the Cayenne maintained-aggregate soundness test
grew to gigabytes and never finished.
CayenneAccelerationExec, the DuckDB aggregate pushdown marker and
PartitionedUnionExec now rebuild the projection over their inner plan (whose
schema they share) before delegating. Regression tests cover the Cayenne
wrapper and the DuckDB marker.
* fix(cayenne): read legacy inline map data under Arrow 59 and keep the type policy
- Inline data written under a nullable map-entries declaration is read through
arrow_tools::map_entries, since Arrow 59 refuses that declaration at decode.
- The primary-key fast-path test's control query no longer uses a bare SUM,
which DataFusion 55 answers from statistics without a scan.
- RunEndEncoded columns stay refused: the Vortex pin can store them, but
accepting them is a product change, so the drift test records the exception.
* fix: bring the remaining tests and connectors in line with DataFusion 55 and Arrow 59
- Flight: repair a server's nullable map-entries declaration before decoding
(Arrow 59 refuses it), as the Flight SQL path does.
- Federation: SQLite and MySQL refuse a filter on a volatile projection output
(DataFusion fork unparser), Turso forwards it, MSSQL opts out.
- Declared types: parse the decimal form directly so an out-of-range
precision/scale still names the problem under arrow-rs's stricter parser.
- Expectations that DataFusion 55 changed: predicate reordering (reorder_q8),
binary literals rendered as X'..', sorts moved below projections (vector
scans), the NSQL context's version and Spark function list, and the
null-aware anti join corrected in Ballista's distributed planner.
- json_get tests follow datafusion-functions-json 0.55 (negative array
indices, json_as_text for a string cast, integral floats in json_get_int).
* fix(vortex): port the merged point-lookup and key-block code to DataFusion 55 and Vortex 0.86
Trunk's point-lookup and key-block changes were written against DataFusion 54 and
the earlier Vortex pin. The scan now binds its projection and filter before
handing them to Vortex (the reader takes bound expressions), the point read keeps
trunk's optimize-then-bind of its conjuncts, dynamic filters are detected through
DynamicFilterTracking, and TableSchema is built with From. Two merged tests move
to the current Vortex Arrow conversion and DataFusion's non-zero batch_size.
* fix(arrow_tools): keep the producer's dictionary ids when repairing a map declaration
Repairing a nullable map-entries declaration re-encoded the IPC schema, and
re-encoding renumbers dictionaries from zero while the dictionary batches that
follow keep the producer's ids. A producer that numbers them differently then
failed to decode, or decoded each dictionary column against another's dictionary
and returned wrong values with no error (reproduced on the Flight SQL read, the
Flight read and the Flight subscribe paths).
The repair now patches the schema message in place: one byte per map field (the
entries nullability, or the Map-to-List type tag for the decodable form),
verified with the flatbuffers verifier and checked against the expected schema;
an unexpected layout is refused rather than passed through.
* fix(runtime): keep the built-in semantics of functions datafusion-spark 55 would shadow
datafusion-spark 55 registers Spark versions of power/pow, atan2 and concat_ws
over the built-ins (infinity instead of an error for zero to a negative power,
Float64 widening, array flattening) and adds hypot, monthname, quote and weekday.
None of that is a deliberate change, so those functions join trunc, date_trunc and
date_part in the withheld list, and a test pins exactly which names resolve to a
Spark implementation. The NSQL context now lists the functions the session
actually registers, so it no longer advertises the withheld ones.
* fix(duckdb): keep a binary literal out of SQL sent to DuckDB
DataFusion 55's unparser renders a binary literal as X'ff', which DuckDB 1.4.4
reads as the text 'xff': through DuckDB federation, b = X'ff' returned the row
holding the bytes 'xff' instead of 0xFF (and X'' missed the empty blob), with no
error. Expressions with a binary literal now stay local; a NULL still pushes
down. Covered end to end by duckdb_binary_literal_filters_agree_with_local_evaluation.
* build: pin DataFusion at 006f3d21, which drops input sums from join statistics
DataFusion 55 answers a bare SUM from exact statistics, and inner/outer join
statistics carried each input's sum through, so SUM over a join returned the whole
input table's sum (silent wrong results; caught by the Cayenne result-correctness
parity tests). The fork fix is spiceai/datafusion#239; the ledger records it with
those parity tests as its guard.
* test(runtime-datafusion): drop a redundant clone in the binary-literal test
* test: fix the Spark built-ins table width and report the pinned DataFusion revision
- the_built_session_keeps_the_built_in_math_and_string_functions: the expected
table padded one column a character wider than the rendered output; the values
were already right.
- spice-substrait-compliance reports DATAFUSION_FORK_REV, which still named the
pre-upgrade revision; it now matches the workspace pin, as its test requires.
* test(oracle): re-record the pushdown plan for DataFusion 55's decimal literal display
DataFusion 55 prints a decimal literal as Decimal128(123.45,10,2) where 54 printed
Decimal128(Some(12345),10,2). The SQL sent to Oracle is unchanged.
* build: repin the forks to their merged fix PRs and carry spiceai-54 forward
- DataFusion f22d1a74: spiceai-54 forward-merged into the 55 line
(spiceai/datafusion#241), bringing spiceai#237 (a Date32 literal's cast uses the
dialect's date type; SQLite read CAST('…' AS DATE) as a number, so date ranges
pushed to SQLite matched nothing) and metadata-column listing pruning.
- arrow-rs be4918e7, Ballista 7c4d54c2, table-providers 3045960a, iceberg 50e982dc,
arrow-adbc 674d6184: the merged CI fix PRs.
- Guard for spiceai#237: a Date32 range unparsed for SQLite keeps the rows it selects
(runs the SQL on SQLite with --features sqlite); it fails on the previous pin.
- Ledger rows for both; the substrait harness reports the new DataFusion pin.
* build: pin DataFusion at ae719330, the merge of spiceai/datafusion#241
Same tree as f22d1a74 (spiceai-54 forward-merged into the 55 line); now the head
of spiceai-55-patches-2 rather than a temporary branch.
* style: rustfmt the SQLite date-range federation test
* build: pin arrow-rs at its spiceai-59 merge and DataFusion at spiceai-55-patches-3
- arrow-rs: 2e2cc330, the merge of spiceai/arrow-rs#29 into spiceai-59 (same tree as the previous pin).
- DataFusion: fc84f2d1 on spiceai-55-patches-3, which carries the Date32 literal fix (spiceai/datafusion#237) but not the metadata-column listing pruning (spiceai#229); the ledger records that pruning as not carried on 55 until it has a guard here.
- Correct the spark-connect-rs branch annotation to spiceai-59-patches-2.
* build: pin DataFusion at the spiceai-55 merge, and iceberg-rust and snowflake-rs at their arrow-rs bumps
- DataFusion: 02550cf9, the merge of spiceai/datafusion#243 into spiceai-55 (same tree as the previous pin); no longer a temporary pin.
- iceberg-rust: 0284be3d and snowflake-rs: b95a74e8, which move each fork's own arrow-rs pin to 2e2cc330 (Cargo manifests only; Spice builds the same code).
* build: repin ballista, table-providers, iceberg-rust and spark-connect-rs to their branch heads
- datafusion-ballista 557ae4ea: arrow-rs/DataFusion landed pins, and the Rust 1.99 clippy fixes (spiceai#70, spiceai#71, spiceai#72).
- datafusion-table-providers 5f50cfd5 and iceberg-rust a0bd6b04: arrow-rs/DataFusion landed pins (spiceai#82, spiceai#53).
- spark-connect-rs 3bf6421a: regenerated lockfile.
datafusion-federation and duckdb-rs stay at 9ca84a39 and 8ee43073: table-providers still pins exactly those revisions, and a second revision of either would put two copies of the crate in the graph. Their newer heads only change their own Cargo manifests and lockfile.
* build: pin iceberg-rust, snowflake-rs, spark-connect-rs, spice-rs and spicebench at their merges
The fork PRs these pins pointed at have merged, mostly as squash merges, so
several pinned commits are no longer on their base branches. Each pin now
points at the merge on the fork's base branch:
- iceberg-rust spiceai-0.10.1-df-55 @ 3e2a14e (spiceai/iceberg-rust#50, spiceai#54)
- snowflake-rs spiceai-59 @ f555738 (spiceai/snowflake-rs#13, spiceai#15)
- spark-connect-rs spiceai-59-2 @ 18ae9bd (spiceai/spark-connect-rs#15, spiceai#16)
- spice-rs trunk @ 4429f39 (spiceai/spice-rs#103)
- spicebench trunk @ 8a90555 (spiceai/spicebench#284)
datafusion-federation stays at 9ca84a3, which is on spiceai-55 because
spiceai/datafusion-federation#88 was a merge commit; only its branch note
changes.
What changes for Spice: lint-only rewrites in snowflake-api, and tests in
iceberg-datafusion and spark-connect-core. The other differences are in
the forks' own [patch.crates-io] sections, which cargo ignores for a
dependency. spice-rs's tree is identical.
* build: pin the forks at their landed version-branch revisions
- arrow-adbc 6e4119ac (spiceai-24, spiceai#7), iceberg-rust 3e2a14e8 (spiceai-0.10.1-df-55, spiceai#50),
datafusion-table-providers 465926a3 (spiceai-55, spiceai#80), snowflake-rs f5557381 (spiceai-59, spiceai#13),
spark-connect-rs 18ae9bd3 (spiceai-59-2, spiceai#15), spice-rs 4429f395 and spicebench 8a90555d (trunk).
- datafusion-federation stays at 9ca84a39, now on spiceai-55 (spiceai#88): datafusion-table-providers pins
exactly that revision.
- duckdb-rs stays at 8ee43073 and temporary: spiceai#49 was squash-merged, so that revision is not on
spiceai-1.4.4, and datafusion-table-providers pins it, so both have to move together.
- datafusion-ballista is unchanged; its version PR is still open.
* build: pin datafusion-ballista at upstream 55.0.0-rc1
spiceai/datafusion-ballista#73 merged upstream's 55.0.0-rc1 tag into
spiceai-55-patches-2, the head of spiceai/datafusion-ballista#68. spiceai#68 is
still open, waiting on license-compliance decisions. Pin 718429f, the spiceai#73
merge, until spiceai#68 lands on spiceai-55.
ballista-core, -executor, -scheduler, -history and -api-types move from
54.0.0 to 55.0.0; no packages are added or removed in Cargo.lock.
`cargo check -p spiced --locked` passes on rustc 1.98.1.
* docs(fork_patches): record the datafusion-ballista pin at 718429f4 (upstream 55.0.0-rc1, spiceai#73)
* build: pin datafusion-ballista at a7c4c585, the merge of spiceai/datafusion-ballista#68 into spiceai-55
* build: pin datafusion-ballista at the spiceai-55 merge of spiceai#68spiceai/datafusion-ballista#68 merged into spiceai-55 as a7c4c58, a merge
commit whose tree is identical to 718429f, the previous pin
(`git diff 718429f a7c4c58` is empty). Repoint the pin at the landed
revision and drop the TEMPORARY marker from the fork-patch ledger, as
scripts/check_fork_patches.py asks. The ledger's ballista section now
names spiceai-55 and upstream 55.0.0-rc1 as its base.
Cargo.lock changes only the source revision of the five ballista packages.
* perf(cayenne): keep small Vortex scans whole on the main query path
DataFusion 55 lowered `repartition_file_min_size` from 10 MiB to 1 MiB
(apache/datafusion#22439), so the main Cayenne scan byte-range-splits a 1-10 MiB
dimension table `target_partitions` ways and pays a Vortex footer open per range.
The main scan now opts small scans out below 10 MiB, as the internal and
protected-snapshot scans already do below 256 MiB; Parquet listing scans keep the
new default. `small_snapshot_groups_opt_out_of_repartitioning` now asserts the
main scan's opt-out and fails with the threshold back at 0.
* docs(fork_patches): record duckdb-rs on the protected spiceai-1.4.4-patches-2 head, which table-providers pins
* test(duckdb): build CreateExternalTable with DataFusion 55's locations field
* ci: validate the datafusion-table-providers pin against spiceai-55, where spiceai#80 landed
* test(cayenne): size the point-lookup fan-out control above the main scan's 10 MiB opt-out
* fix(data-accelerator-api): implement DataFusion 55's apply_expressions for KeepFirstExec
* test: port trunk's new tests to DataFusion 55 (CreateExternalTable locations, common::TableReference)
* fix(cayenne): keep -0.0 and 0.0 equal in pushed-down float comparisons and memory-mode indexes
DataFusion 55 compares floats with the two zeros equal (apache/datafusion#22835),
while Vortex and the Arrow row encoding order them apart. Spell out both zeros
when a float comparison or IN list with a zero literal is pushed into a Vortex
scan, keep float column-vs-column comparisons above the scan, and hash
memory-mode index keys with the file-mode KeyEncoder.
* Fix lint
---------
Co-authored-by: Luke Kim <80174+lukekim@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves the Spice iceberg-rust 0.10.1 line to DataFusion 55 and Arrow 59 (
spiceai-0.10.1-df-55, cut fromspiceai-0.10.1-df-54).serde_arrowarrow-59.PyCapsule::new_with_value(PyO3 0.29 deprecation).Pinned by spiceai/spiceai#14612 (tracking: spiceai/spiceai#13570). Supersedes #47.
CI test fixtures (backported in #51)
The Tests job could not start the integration fixtures on this line:
make docker-upfailed before any test ran. #51 cherry-picks two upstream CI fixes. Both are test-infrastructure changes, independent of the DataFusion 55 / Arrow 59 move:pull access denied for minio/mc). The test helperget_minio_endpointbecomesget_object_store_endpointwith it. The one conflict, indev/docker-compose.yaml, keeps this line's REST fixtureCATALOG_URI.404 Not Foundfromdebian-security(run 36816859585).Validation: Tests (default) on a0bd6b0 ran
1859 tests run: 1859 passed, 1 skipped(run 36962200040).