RecordBatchTransformer does not apply initial-default for nested fields #3261

Is your feature request related to a problem or challenge?

Problem - regarding an edge case with nested struct fields and Iceberg's initial-default option in the spec

Describe the solution you'd like

RecordBatchTransformer null-fills a nested field that is missing from an older data file. If that field has an initial-default, it returns FeatureUnsupported instead of applying the default:

initial-default of nested field {name} is not supported

Top-level fields already apply initial-default through ColumnSource::Add, but nested fields don't.

The spec tracks defaults on each field, including fields inside a struct. A parent struct's default must not embed its children's defaults. The effective value is built by applying each field's own default.

From the spec's point example: point.y has initial-default 0, and an older file has {"x": 3}. The read should return {"x": 3, "y": 0}, not null for y.

https://iceberg.apache.org/spec/#default-values

This shows up on v3 tables when a nested field was added with a non-null initial-default and older files are still in the snapshot. Those reads fail until this is implemented, which is preferable to silently returning nulls.

(Created to track as for the future since it's out of scope for #3255)

Willingness to contribute

I can contribute to this feature independently