ci: replace MinIO with RustFS for integration tests by comphead · Pull Request #3273 · apache/iceberg-rust

comphead · 2026-09-24T17:07:54Z

Which issue does this PR close?

Closes ci: Fix MinIO references #3272.

What changes are included in this PR?

quay.io/minio/minio no longer allows anonymous pulls (see apache/datafusion#25705), so make docker-up fails and the Start Docker containers step of Tests (default) has been failing since 2026-09-24. DataFusion fixed the same breakage by switching to RustFS in apache/datafusion#25706, and PyIceberg moved to RustFS in apache/iceberg-python#3928. This PR does the same here.

Following PyIceberg, the S3 service uses the vendor-neutral name object-store, so a future backend swap only needs an image change.

In dev/docker-compose.yaml:

The minio service becomes object-store, running rustfs/rustfs:1.0.0. The admin/password credentials, the 9000 (S3) and 9001 (console) ports, and virtual-hosted-style access through RUSTFS_SERVER_DOMAINS=object-store and the *.object-store network aliases stay the same.
The healthcheck uses /health/ready, which returns 200 only once storage and IAM are ready. /health is a liveness probe.
The mc service becomes create-buckets, a one-shot job that reuses the RustFS image and creates icebergdata, warehouse, and bucket1 with chained curl -f --aws-sigv4 calls, as the DataFusion PR does. Any failure fails docker compose up --wait. Re-creating an existing bucket returns 200, so it is safe to re-run. spark-iceberg waits for it to complete successfully, so provisioning cannot race bucket creation. The mc policy set public calls are dropped because no test accesses the buckets anonymously.

The REST fixture, dev/hms/core-site.xml, and dev/spark/spark-defaults.conf now point at http://object-store:9000. The test helpers are renamed to match: get_minio_endpoint becomes get_object_store_endpoint and ICEBERG_TEST_MINIO_ENDPOINT becomes ICEBERG_TEST_OBJECT_STORE_ENDPOINT.

make docker-up now passes --remove-orphans, so a stack started before the rename doesn't leave a stale minio container holding port 9000.

The minio:// examples in crates/storage/opendal/src/lib.rs are unchanged. They describe custom scheme support, not the test setup.

Are these changes tested?

Covered by the existing integration tests, which run against RustFS in CI via make docker-up.

AI Disclosure

Prepared with Claude Code (Claude Opus 5.5), modeled on apache/datafusion#25706. Follow-up commits (vendor-neutral naming, one-shot bucket creation, --remove-orphans) prepared with GitHub Copilot.

`quay.io/minio/minio` no longer allows anonymous pulls, so `make docker-up` fails and the integration tests cannot start. DataFusion hit the same failure and switched to RustFS in apache/datafusion#25706. PyIceberg moved to RustFS earlier in apache/iceberg-python#3928. Run `rustfs/rustfs:1.0.0` as the shared S3 service instead. It keeps the `admin`/`password` credentials, the 9000/9001 ports, and virtual-hosted-style access through `RUSTFS_SERVER_DOMAINS` and the bucket aliases. The healthcheck uses `/health/ready`, which waits for storage and IAM. The `mc` bucket setup becomes `curl --aws-sigv4` run from the RustFS image, and the public bucket policy is dropped because no test reads anonymously. Point the REST fixture, HMS, and Spark at `http://rustfs:9000`, and rename the MinIO-specific test helpers and the `ICEBERG_TEST_MINIO_ENDPOINT` override to RustFS. Closes apache#3272

Copilot

Copilot review overview

🟡 Changes recommended

Ensure bucket provisioning failures cause the Compose service to fail.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

Ensure bucket provisioning fails when curl requests fail · New

What changed in this PR

Replaces MinIO with RustFS for integration-test object storage and updates related configurations, helpers, tests, and documentation.

Changes:

Switches Docker Compose services and bucket provisioning to RustFS.
Updates HMS, Spark, REST, and test endpoints.
Renames RustFS test helpers and documents the runtime change.

File	Reviewed changes
`website/src/reference/container-runtimes.md`	Updates runtime documentation.
`dev/spark/spark-defaults.conf`	Points Spark storage to RustFS.
`dev/hms/core-site.xml`	Points HMS storage to RustFS.
`dev/docker-compose.yaml`	Defines RustFS and bucket provisioning; provisioning failures should stop the service.
`crates/test_utils/src/lib.rs`	Renames endpoint helpers and environment variables.
`crates/storage/opendal/tests/resolving_storage_test.rs`	Updates resolving-storage tests.
`crates/storage/opendal/tests/file_io_s3_test.rs`	Updates S3 test configuration.
`crates/integration_tests/src/lib.rs`	Uses the RustFS endpoint.
`crates/catalog/loader/tests/common/mod.rs`	Updates catalog test configuration.
`crates/catalog/hms/tests/hms_catalog_test.rs`	Uses RustFS in HMS tests.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+        curl -fsS --aws-sigv4 aws:amz:us-east-1:s3 -u admin:password -X PUT http://rustfs:9000/icebergdata;
+        curl -fsS --aws-sigv4 aws:amz:us-east-1:s3 -u admin:password -X PUT http://rustfs:9000/warehouse;
+        curl -fsS --aws-sigv4 aws:amz:us-east-1:s3 -u admin:password -X PUT http://rustfs:9000/bucket1;


Rename the RustFS service, hostname, bucket aliases, test helpers and env var to a generic object-store name (matching PyIceberg), so future backend swaps only need an image change. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Treat any non-200/409 response as a failure instead of ignoring it, and add a healthcheck so `docker compose up --wait` blocks until the buckets exist. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Renaming the minio service leaves stale containers holding port 9000 for anyone with a previously started stack; clean them up automatically. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

RustFS returns 200 when re-creating an existing bucket, so plain chained curl -f calls are idempotent. Replace the status-code parsing, marker-file healthcheck and tail with a one-shot container, and gate spark-iceberg on its successful completion. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

kevinjqliu

LGTM!

i pushed a fix to rename to object-store instead of rustfs. Just in case we have to change it again in the future 😄 and this aligns with pyiceberg

kevinjqliu · 2026-09-24T18:46:11Z

 # Docker targets for integration tests
 docker-up:
-	docker compose -f dev/docker-compose.yaml up -d --build --wait
+	docker compose -f dev/docker-compose.yaml up -d --build --wait --remove-orphans


in case previous container is still running, its now a different name so hence orphaned

kevinjqliu · 2026-09-24T18:48:27Z

quay.io/minio/minio no longer allows anonymous pulls (see apache/datafusion#25705)

oh boy. thanks for the heads up. we just moved all minio to from docker hub to quay.io as a hot fix. now we gotta move everything again to rustfs

kevinjqliu · 2026-09-24T18:57:24Z

we should just switch all the iceberg repos over, created a tracking issue: apache/iceberg#18246

kevinjqliu · 2026-09-24T18:57:33Z

Thanks again @comphead

comphead · 2026-09-24T21:08:41Z

Thanks @kevinjqliu for the review

* ci: replace MinIO with RustFS for integration tests `quay.io/minio/minio` no longer allows anonymous pulls, so `make docker-up` fails and the integration tests cannot start. DataFusion hit the same failure and switched to RustFS in apache/datafusion#25706. PyIceberg moved to RustFS earlier in apache/iceberg-python#3928. Run `rustfs/rustfs:1.0.0` as the shared S3 service instead. It keeps the `admin`/`password` credentials, the 9000/9001 ports, and virtual-hosted-style access through `RUSTFS_SERVER_DOMAINS` and the bucket aliases. The healthcheck uses `/health/ready`, which waits for storage and IAM. The `mc` bucket setup becomes `curl --aws-sigv4` run from the RustFS image, and the public bucket policy is dropped because no test reads anonymously. Point the REST fixture, HMS, and Spark at `http://rustfs:9000`, and rename the MinIO-specific test helpers and the `ICEBERG_TEST_MINIO_ENDPOINT` override to RustFS. Closes apache#3272 * ci: use vendor-neutral object-store naming for S3 test service Rename the RustFS service, hostname, bucket aliases, test helpers and env var to a generic object-store name (matching PyIceberg), so future backend swaps only need an image change. * ci: fail fast and gate readiness on test bucket creation Treat any non-200/409 response as a failure instead of ignoring it, and add a healthcheck so `docker compose up --wait` blocks until the buckets exist. * ci: remove orphan containers on docker-up Renaming the minio service leaves stale containers holding port 9000 for anyone with a previously started stack; clean them up automatically. * ci: make create-buckets a one-shot job RustFS returns 200 when re-creating an existing bucket, so plain chained curl -f calls are idempotent. Replace the status-code parsing, marker-file healthcheck and tail with a one-shot container, and gate spark-iceberg on its successful completion. --------- Co-authored-by: Oleks V <comphead@users.noreply.github.com> Co-authored-by: ovoievodin <o_voievodin@apple.com> Co-authored-by: Kevin Liu <kevin.jq.liu@gmail.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* ci: replace MinIO with RustFS for integration tests `quay.io/minio/minio` no longer allows anonymous pulls, so `make docker-up` fails and the integration tests cannot start. DataFusion hit the same failure and switched to RustFS in apache/datafusion#25706. PyIceberg moved to RustFS earlier in apache/iceberg-python#3928. Run `rustfs/rustfs:1.0.0` as the shared S3 service instead. It keeps the `admin`/`password` credentials, the 9000/9001 ports, and virtual-hosted-style access through `RUSTFS_SERVER_DOMAINS` and the bucket aliases. The healthcheck uses `/health/ready`, which waits for storage and IAM. The `mc` bucket setup becomes `curl --aws-sigv4` run from the RustFS image, and the public bucket policy is dropped because no test reads anonymously. Point the REST fixture, HMS, and Spark at `http://rustfs:9000`, and rename the MinIO-specific test helpers and the `ICEBERG_TEST_MINIO_ENDPOINT` override to RustFS. Closes apache#3272 * ci: use vendor-neutral object-store naming for S3 test service Rename the RustFS service, hostname, bucket aliases, test helpers and env var to a generic object-store name (matching PyIceberg), so future backend swaps only need an image change. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * ci: fail fast and gate readiness on test bucket creation Treat any non-200/409 response as a failure instead of ignoring it, and add a healthcheck so `docker compose up --wait` blocks until the buckets exist. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * ci: remove orphan containers on docker-up Renaming the minio service leaves stale containers holding port 9000 for anyone with a previously started stack; clean them up automatically. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * ci: make create-buckets a one-shot job RustFS returns 200 when re-creating an existing bucket, so plain chained curl -f calls are idempotent. Replace the status-code parsing, marker-file healthcheck and tail with a one-shot container, and gate spark-iceberg on its successful completion. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: ovoievodin <o_voievodin@apple.com> Co-authored-by: Kevin Liu <kevin.jq.liu@gmail.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Replace deprecated minio/minio and minio/mc images with rustfs/rustfs:1.0.0, matching the migration in apache/iceberg-rust#3273 and risingwavelabs/iceberg-rust#249.

* ci: replace MinIO with RustFS for integration tests `quay.io/minio/minio` no longer allows anonymous pulls, so `make docker-up` fails and the integration tests cannot start. DataFusion hit the same failure and switched to RustFS in apache/datafusion#25706. PyIceberg moved to RustFS earlier in apache/iceberg-python#3928. Run `rustfs/rustfs:1.0.0` as the shared S3 service instead. It keeps the `admin`/`password` credentials, the 9000/9001 ports, and virtual-hosted-style access through `RUSTFS_SERVER_DOMAINS` and the bucket aliases. The healthcheck uses `/health/ready`, which waits for storage and IAM. The `mc` bucket setup becomes `curl --aws-sigv4` run from the RustFS image, and the public bucket policy is dropped because no test reads anonymously. Point the REST fixture, HMS, and Spark at `http://rustfs:9000`, and rename the MinIO-specific test helpers and the `ICEBERG_TEST_MINIO_ENDPOINT` override to RustFS. Closes apache#3272 * ci: use vendor-neutral object-store naming for S3 test service Rename the RustFS service, hostname, bucket aliases, test helpers and env var to a generic object-store name (matching PyIceberg), so future backend swaps only need an image change. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * ci: fail fast and gate readiness on test bucket creation Treat any non-200/409 response as a failure instead of ignoring it, and add a healthcheck so `docker compose up --wait` blocks until the buckets exist. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * ci: remove orphan containers on docker-up Renaming the minio service leaves stale containers holding port 9000 for anyone with a previously started stack; clean them up automatically. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * ci: make create-buckets a one-shot job RustFS returns 200 when re-creating an existing bucket, so plain chained curl -f calls are idempotent. Replace the status-code parsing, marker-file healthcheck and tail with a one-shot container, and gate spark-iceberg on its successful completion. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: ovoievodin <o_voievodin@apple.com> Co-authored-by: Kevin Liu <kevin.jq.liu@gmail.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 119fad9)

Copilot AI lite review requested due to automatic review settings September 24, 2026 17:07

Copilot started reviewing on behalf of comphead September 24, 2026 17:12 View session

Copilot AI reviewed Sep 24, 2026

View reviewed changes

kevinjqliu added the datafusion label Sep 24, 2026

kevinjqliu and others added 4 commits September 24, 2026 11:28

ci: fail fast and gate readiness on test bucket creation

6884398

Treat any non-200/409 response as a failure instead of ignoring it, and add a healthcheck so `docker compose up --wait` blocks until the buckets exist. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

ci: remove orphan containers on docker-up

21f163c

Renaming the minio service leaves stale containers holding port 9000 for anyone with a previously started stack; clean them up automatically. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

kevinjqliu approved these changes Sep 24, 2026

View reviewed changes

kevinjqliu enabled auto-merge September 24, 2026 18:47

kevinjqliu added this pull request to the merge queue Sep 24, 2026

kevinjqliu mentioned this pull request Sep 24, 2026

Replace MinIO with RustFS across Iceberg repos apache/iceberg#18246

Closed

3 tasks

Merged via the queue into apache:main with commit 119fad9 Sep 24, 2026
25 checks passed

kevinjqliu mentioned this pull request Sep 25, 2026

ci: enable RustFS stdout logging in docker compose #3279

Draft

chenzl25 mentioned this pull request Sep 25, 2026

ci: replace MinIO with RustFS for integration tests (cherry-pick #3273) risingwavelabs/iceberg-rust#249

Merged

xanderbailey mentioned this pull request Sep 28, 2026

0.11.x backport: ci: fix Rust cache action casing (#3249) + replace MinIO with RustFS (#3273) #3271

Open

Navigation Menu

ci: replace MinIO with RustFS for integration tests - #3273

ci: replace MinIO with RustFS for integration tests#3273
kevinjqliu merged 5 commits into
apache:mainfrom
comphead:replace-minio-with-rustfs

comphead commented Sep 24, 2026 •

edited by kevinjqliu

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

kevinjqliu left a comment

Uh oh!

kevinjqliu Sep 24, 2026

Uh oh!

kevinjqliu commented Sep 24, 2026

Uh oh!

kevinjqliu commented Sep 24, 2026

Uh oh!

kevinjqliu commented Sep 24, 2026

Uh oh!

Uh oh!

comphead commented Sep 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

comphead commented Sep 24, 2026 • edited by kevinjqliu Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

What changes are included in this PR?

Are these changes tested?

AI Disclosure

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

kevinjqliu left a comment

Choose a reason for hiding this comment

Uh oh!

kevinjqliu Sep 24, 2026

Choose a reason for hiding this comment

Uh oh!

kevinjqliu commented Sep 24, 2026

Uh oh!

kevinjqliu commented Sep 24, 2026

Uh oh!

kevinjqliu commented Sep 24, 2026

Uh oh!

Uh oh!

comphead commented Sep 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

comphead commented Sep 24, 2026 •

edited by kevinjqliu

Loading