DataFrame API: allow aggregate functions in select() 2 (#17874)

[cj-zhukov]

Which issue does this PR close?

• Closes DataFrame API: allow aggregate functions in select() #17874.

Rationale for this change

DataFrame::select currently does not support aggregate expressions directly. Users must explicitly call DataFrame::aggregate. This PR improves ergonomics by allowing aggregate expressions inside select, while preserving existing behavior and validation rules.

What changes are included in this PR?

• DataFrame::select lifts a global Aggregate when every expression is an aggregate or a scalar over aggregates (and literals).
• Existing tests were not rewritten.
• Examples are left for a follow-up.

What is the testing strategy for this PR?

• Existing DataFrame select/aggregate tests continue to pass without edits.
• Added test_dataframe_api_select_semantics (from DataFrame API: allow aggregate functions in select() (#17874) #21021, plus two cases for this design).
• roundtrip_expr_api was not changed and still passes (mixed aggregate + scalar lists stay a Projection).

Are there any user-facing changes?

Yes — additive behavior only. The public select() signature is unchanged. There are no breaking API changes.

Users can write a global aggregation with select():

ctx.table("aggregate_test_100")
    .await?
    .select(vec![
        approx_distinct(col("c9")).alias("count_c9"),
        approx_distinct(cast(col("c9"), DataType::Utf8View))
            .alias("count_c9_str"),
    ])?
    .show()
    .await?;

Previously this required:

ctx.table("aggregate_test_100")
    .await?
    .aggregate(
        vec![],
        vec![
            approx_distinct(col("c9")).alias("count_c9"),
            approx_distinct(cast(col("c9"), DataType::Utf8View))
                .alias("count_c9_str"),
        ],
    )?
    .show()
    .await?;

Both remain valid. Grouped aggregation is still aggregate(group, aggs).

[codecov-commenter]

Codecov Report

❌ Patch coverage is 87.12871% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.73%. Comparing base (c149764) to head (197d541).
⚠️ Report is 348 commits behind head on main.

Files with missing lines	Patch %	Lines
datafusion/core/src/dataframe/mod.rs	87.12%	3 Missing and 10 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #25482      +/-   ##
==========================================
+ Coverage   81.91%   82.73%   +0.81%     
==========================================
  Files        1134     1147      +13     
  Lines      425703   449552   +23849     
  Branches   425703   449552   +23849     
==========================================
+ Hits       348725   371937   +23212     
+ Misses      56300    54942    -1358     
- Partials    20678    22673    +1995     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[cj-zhukov]

hey @alamb @Jefffrey @martin-g
could you please take a look at this PR? We were discussing the same issue in #21021 but it was closed because of unwanted breaking changes. I took another step without breaking changes now.

[kosiew]

@cj-zhukov,

Thanks for working on this. The aggregate support looks useful, but I think the new output-name deduplication changes an existing validation rule, so I'd like that addressed before approval.

[kosiew]

This now silently renames duplicate explicit aliases before the existing projection validator can reject them, so select behaves differently from ordinary projections and explicit aggregate. Please remove the public output-name uniquification and let the existing projection validation reject duplicate names, while keeping unique internal aggregate aliases where needed.

[cj-zhukov]

I agree with you

[kosiew]

Could you also add a small empty-input case using filter(lit(false)).select(...) with distinct aliases? It would be useful to verify that global aggregation still returns exactly one row, with COUNT = 0, SUM = NULL, and the literal value preserved.

[cj-zhukov]

Agreed. Let me add new tests.

[cj-zhukov]

@kosiew Thank you for the valuable review!

I've removed the public output-name uniquification and added empty-input test case. Could you please take another look when you have a chance?

[kosiew]

@cj-zhukov,

Thanks for addressing the review comments. The follow-up restores the existing duplicate projection name validation while keeping internal aggregate aliases unique. The new empty-input test also verifies the expected global aggregation behavior.

I've reviewed the changes and have no further concerns. LGTM!