Skip to content

No per-table credential vending in REST catalog #10

Description

@smaheshwar-pltr

Problem

iceberg-cpp's REST catalog has no support for per-table credential vending. When a REST catalog server returns vended credentials (temporary S3 access keys) in the LoadTableResponse, iceberg-cpp ignores them and uses the catalog-level FileIO for all tables. This means consumers built on iceberg-cpp cannot access tables in environments where the REST catalog vends per-table scoped S3 credentials.

Consumers using iceberg-cpp to read tables from a credential-vending REST catalog get authentication failures when trying to read data files, because no S3 credentials are propagated from the LoadTableResponse to the table's FileIO.

Error

When scanning a table whose data files require vended credentials:

HTTP 403 Forbidden
AccessDenied: Access Denied
Authentication Failure - this is usually caused by invalid or missing credentials.
* No credentials are provided.

The catalog discovery and table metadata loading succeed (those use the catalog-level credentials), but reading the actual Parquet/Avro data files fails because the per-table S3 credentials from the REST catalog response are never extracted or applied.

Expected behavior

Per the Iceberg REST Catalog spec, when X-Iceberg-Access-Delegation: vended-credentials is sent, the server returns temporary S3 credentials in LoadTableResponse.config. These should be:

  1. Extracted from the response
  2. Merged with catalog properties (catalog < table config < storage credentials, with longest prefix match)
  3. Used to create a per-table FileIO
  4. Accessible to consumers via FileIO::properties() (matching the pattern in iceberg-java and PyIceberg)

Reference implementations

  • Java: RESTSessionCatalog sends X-Iceberg-Access-Delegation and resolves per-table FileIO. Trino reads credentials via baseTable.io().properties().
  • Python: PyIceberg's REST catalog merges vended credentials into table.io.properties for engine consumption.
  • Rust: Credentials flow through table.file_io().config().props().

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions