fix: preserve literal metadata and destructure leaf and unary proto hooks

[peterxcli]

Which issue does this PR close?

Closes #24613. Literal field metadata is lost when physical expressions are serialized.

Rationale for this change

Losing this metadata can strip Arrow extension types and custom metadata from projected columns.

What changes are included in this PR?

Use exhaustive destructuring in all seven leaf and unary expression encoders and decoders. Add a literal protobuf message that carries metadata and regenerate the Rust and JSON bindings.

What is the testing strategy for this PR?

A projection round-trip test covers null and non-null literals through protobuf and JSON and was verified to fail before the fix. Existing tests cover plain literals, and a new test rejects messages missing a value. The extended workspace tests and all-feature Clippy pass.

Are there any user-facing changes?

Literal metadata survives physical plan serialization. Literals with metadata use a new protobuf variant that requires an updated reader.

[github-actions]

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details

     Cloning apache/main
    Building datafusion-physical-expr v55.1.0 (current)
       Built [  22.641s] (current)
     Parsing datafusion-physical-expr v55.1.0 (current)
      Parsed [   0.039s] (current)
    Building datafusion-physical-expr v55.1.0 (baseline)
       Built [  22.160s] (baseline)
     Parsing datafusion-physical-expr v55.1.0 (baseline)
      Parsed [   0.038s] (baseline)
    Checking datafusion-physical-expr v55.1.0 -> v55.1.0 (no change; assume patch)
     Checked [   0.425s] 223 checks: 223 pass, 31 skip
     Summary no semver update required
    Finished [  46.797s] datafusion-physical-expr
    Building datafusion-proto v55.1.0 (current)
       Built [  40.053s] (current)
     Parsing datafusion-proto v55.1.0 (current)
      Parsed [   0.011s] (current)
    Building datafusion-proto v55.1.0 (baseline)
       Built [  41.044s] (baseline)
     Parsing datafusion-proto v55.1.0 (baseline)
      Parsed [   0.011s] (baseline)
    Checking datafusion-proto v55.1.0 -> v55.1.0 (no change; assume patch)
     Checked [   0.140s] 223 checks: 223 pass, 31 skip
     Summary no semver update required
    Finished [  82.461s] datafusion-proto
    Building datafusion-proto-models v55.1.0 (current)
       Built [  19.498s] (current)
     Parsing datafusion-proto-models v55.1.0 (current)
      Parsed [   0.111s] (current)
    Building datafusion-proto-models v55.1.0 (baseline)
       Built [  19.511s] (baseline)
     Parsing datafusion-proto-models v55.1.0 (baseline)
      Parsed [   0.112s] (baseline)
    Checking datafusion-proto-models v55.1.0 -> v55.1.0 (no change; assume patch)
     Checked [   2.365s] 223 checks: 222 pass, 1 fail, 0 warn, 31 skip

--- failure enum_variant_added: enum variant added on exhaustive enum ---

Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/enum_variant_added.ron

Failed in:
  variant ExprType:LiteralWithMetadata in /home/runner/work/datafusion/datafusion/datafusion/proto-models/src/generated/prost.rs:1704
  variant ExprType:LiteralWithMetadata in /home/runner/work/datafusion/datafusion/datafusion/proto-models/src/generated/prost.rs:1704

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [  42.783s] datafusion-proto-models

[codecov-commenter]

Codecov Report

❌ Patch coverage is 77.24138% with 33 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.67%. Comparing base (76f9fde) to head (c7d4b0b).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
datafusion/proto-models/src/generated/pbjson.rs	67.60%	12 Missing and 11 partials ⚠️
datafusion/physical-expr/src/expressions/column.rs	64.28%	5 Missing ⚠️
...atafusion/physical-expr/src/expressions/literal.rs	89.47%	0 Missing and 4 partials ⚠️
datafusion/proto/src/physical_plan/from_proto.rs	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #25382      +/-   ##
==========================================
+ Coverage   82.65%   82.67%   +0.02%     
==========================================
  Files        1147     1147              
  Lines      446087   446205     +118     
  Branches   446087   446205     +118     
==========================================
+ Hits       368710   368899     +189     
+ Misses      54990    54857     -133     
- Partials    22387    22449      +62     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[kosiew]

@peterxcli,

Thanks for working on this. The change preserves literal field metadata across physical expression protobuf and JSON round trips while keeping the existing encoding for literals without metadata. The leaf and unary proto hook destructuring also looks behavior-preserving. I did not find any issues that need changes.