feat: derive per-column pruning guarantees from tuple IN lists

[yashrb24]

Which issue does this PR close?

Closes #25463

Rationale for this change

Multi-column joins can produce filters such as:

(a, b) IN ((1, 10), (2, 20))

DataFusion can evaluate this filter on individual rows, but it does not extract the allowed values for each column. This prevents bloom filters from using those values to skip row groups.

What changes are included in this PR?

• Extract per-column values: a IN (1, 2) and b IN (10, 20).
• Keep the original tuple filter, so combinations such as (1, 20) are still rejected.

This will only enable bloom filter pruning for row groups.

What is the testing strategy for this PR?

Added tests for value extraction, NULLs, dictionary values, and reordered named fields.

Are there any user-facing changes?

Queries with supported tuple IN filters, including dynamic filters from multi-column joins, may read fewer row groups when Bloom filters are available. Query results remain unchanged.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 93.25843% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.64%. Comparing base (1e09a2a) to head (f45a5b9).
⚠️ Report is 181 commits behind head on main.

Files with missing lines	Patch %	Lines
datafusion/physical-expr/src/utils/guarantee.rs	92.72%	3 Missing and 9 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #25469      +/-   ##
==========================================
+ Coverage   82.42%   82.64%   +0.22%     
==========================================
  Files        1138     1147       +9     
  Lines      435429   445677   +10248     
  Branches   435429   445677   +10248     
==========================================
+ Hits       358889   368332    +9443     
- Misses      54839    54977     +138     
- Partials    21701    22368     +667     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[yashrb24]

StructFunc from datafusion-functions crate already provides support for this, but instead of adding another dependency I thought it might be a better idea to have it defined separately here instead

[rgbuilds]

The unit test nicely covers both pruning and the crossed-pair correlation case, while the SLT exercises the real Parquet scan path. Would it be valuable to combine these in one end-to-end test with multiple row groups? For example, the test could include:

1. A row group containing an exact tuple match.
2. A row group that statistics cannot eliminate but Bloom filters can, verifying that it is counted as pruned in row_groups_pruned_bloom_filter.
3. A crossed-pair row group that satisfies the derived per-column guarantees and therefore survives Bloom pruning, but produces no rows after evaluation of the original tuple predicate.

This would verify that the derived per-column conditions are used only for pruning, while the original tuple predicate is still applied for exact row filtering.

[yashrb24]

thanks for the review! this makes sense to me, will extend this test

[rgbuilds]

I think duplicate named_struct field names can make this inference unsound.

named_struct currently permits duplicate names and emits a mapping entry for each field, while get_field uses StructArray::column_by_name, which selects the first matching field when names are duplicated.

For example:

named_struct('k', x, 'k', y)
    IN (named_struct('k', 1, 'k', 10))

Both mapping entries evaluate get_field(..., 'k') against the first literal field. This appears to derive:

x IN (1)
y IN (1)

However, (x=1, y=10) satisfies the original tuple predicate.

A row group containing the matching row (1,10) and a filler row (0,0) would have y_min=0 and y_max=10, so statistics cannot eliminate y=1. Its Bloom filter could nevertheless prove that y=1 is absent and incorrectly prune the row group containing the valid (1,10) match.

Would it be safer for named_struct::struct_field_mapping to return no mapping when an accessor name is repeated, with a focused regression test for this case?

[rgbuilds]

Thanks. Returning no mapping when a field name is repeated prevents the ambiguous inference, and the focused regression test covers the case I raised. This resolves my concern.

[rgbuilds]

Thanks for adding the three-row-group end-to-end scan case. This addresses my earlier review comment: the revised test now exercises:

• an exact-match row group,
• a Bloom-pruned row group, and
• a crossed-pair row group that survives the derived per-column guarantees but is rejected by the original tuple predicate.

I also ran test_struct_inlist_guarantees and the push_down_filter_parquet SLT locally; both passed.

While reviewing the revised guarantee-extraction path, I found a duplicate-field-name case in named_struct that appears capable of deriving an invalid necessary guarantee and pruning a row group containing a valid match. I left the concrete counterexample inline.

I’m requesting changes pending resolution of that correctness case.

[rgbuilds]

Thanks for the updates. The duplicate-field-name correctness concern and the earlier end-to-end scan coverage are both addressed. The latest changes look good to me.