Skip to content

feat(io): expose FileIO configuration properties - #14

Open
smaheshwar-pltr wants to merge 2 commits into
mainfrom
rest-fileio-properties
Open

smaheshwar-pltr wants to merge 2 commits into
mainfrom
rest-fileio-properties

Conversation

@smaheshwar-pltr

@smaheshwar-pltr smaheshwar-pltr commented Jun 27, 2026 •

Copy link
Copy Markdown
Owner

What

Add FileIO::properties() and expose configuration from both ArrowS3FileIO and ResolvingFileIO. Other implementations retain an empty default.

Why

Engines that perform their own I/O need the resolved table FileIO configuration, including credentials returned in REST table config. This follows Java FileIO.properties() and PyIceberg's property-map API. Structured storage-credentials remain on SupportsStorageCredentials.

The resolver override is required after the scheme-based FileIO work in apache#828 and apache#889. Because the map may contain credentials, the API contract warns callers not to log or expose it.

Validation

Added S3 and REST resolver coverage, including catalog/table merge precedence. The full GitHub Actions matrix passed.

AI assistance: Codex was used to rebase and adapt the existing change to the current resolver architecture. I reviewed the resulting diff and validation.

Comment thread src/iceberg/arrow/s3/arrow_s3_file_io.cc Outdated
@smaheshwar-pltr
smaheshwar-pltr force-pushed the rest-fileio-properties branch from fc46f62 to 13a013b Compare June 27, 2026 23:21
Comment thread src/iceberg/arrow/s3/arrow_s3_file_io.cc Outdated
@smaheshwar-pltr
smaheshwar-pltr force-pushed the rest-fileio-properties branch 2 times, most recently from fe44eb3 to 20f9c65 Compare June 28, 2026 00:05
@smaheshwar-pltr smaheshwar-pltr changed the title feat(io): expose resolved FileIO properties for a location feat(io): expose FileIO properties Jul 9, 2026
@smaheshwar-pltr smaheshwar-pltr changed the title feat(io): expose FileIO properties feat(io): expose FileIO configuration properties Sep 6, 2026
return storage_credentials_;
}

const std::unordered_map<std::string, std::string>& properties() const override {

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant