Skip to content

parse table metadata.configuration as TableProperties#453

Merged
zachschuermann merged 27 commits into
delta-io:mainfrom
zachschuermann:table-properties
Nov 25, 2024
Merged

parse table metadata.configuration as TableProperties#453
zachschuermann merged 27 commits into
delta-io:mainfrom
zachschuermann:table-properties

Conversation

@zachschuermann
Copy link
Copy Markdown
Collaborator

@zachschuermann zachschuermann commented Nov 5, 2024

What changes are proposed in this pull request?

New TableProperties struct parses the Metadata action's configuration into a strongly-typed configuration struct. Instead of previously manually querying specific keys (e.g. checking ColumnMapping key) we can leverage the configuration to appropriately handle DVs, column mapping, etc. in a structured manner.

Pragmatically, the changes are:

  1. New TableProperties struct along with new Snapshot::get_table_properties() API
    a. Note that every field is optional and additionally there is an unknown_properties field which includes a hashmap of all fields that did not parse to a known table property.
  2. table_properties module including deserialization submodule to implement functions to deserialize the fields of TableProperties

Future work

This PR affects the following public APIs

  • New TableProperties and Snapshot::get_table_properties API

How was this change tested?

new ut

Credit: @roeap for the original PR which this is based on #222

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking-change Public API change that could cause downstream compilation failures. Requires a major version bump.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants