Skip to content

feat: rewrite count(non-nullable column) to count(1) - #25990

Open
wudidapaopao wants to merge 1 commit into
apache:mainfrom
wudidapaopao:optimize-count-non-nullable-column
Open

wudidapaopao wants to merge 1 commit into
apache:mainfrom
wudidapaopao:optimize-count-non-nullable-column

Conversation

@wudidapaopao

@wudidapaopao wudidapaopao commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

COUNT(non_nullable_column) counts every input row, but evaluating it still reads and materializes the column unnecessarily.

What changes are included in this PR?

  • Rewrite safe, non-DISTINCT counts over non-nullable columns to COUNT(1) during expression simplification while preserving output names and schemas.
  • Keep SingleDistinctToGroupBy rewrites working when name preservation adds an alias around the rewritten count.
  • Avoid marking unchanged aggregate simplifications as transformed.

What is the testing strategy for this PR?

Covered by unit, optimizer, column-pruning, and sqllogictest cases.

Are there any user-facing changes?

Query results and output schemas are unchanged. Optimized plans can show count(1) AS count(column), and scans can prune non-nullable columns used only by COUNT.

@github-actions github-actions Bot added optimizer Optimizer rules core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels Oct 3, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.55224% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.62%. Comparing base (b550f59) to head (92fe1b3).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/functions-aggregate/src/count.rs 87.50% 2 Missing and 5 partials ⚠️
...fusion/optimizer/src/single_distinct_to_groupby.rs 86.04% 2 Missing and 4 partials ⚠️
...imizer/src/simplify_expressions/expr_simplifier.rs 97.05% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #25990      +/-   ##
==========================================
- Coverage   82.62%   82.62%   -0.01%     
==========================================
  Files        1147     1147              
  Lines      445272   445397     +125     
  Branches   445272   445397     +125     
==========================================
+ Hits       367913   368013     +100     
- Misses      55016    55028      +12     
- Partials    22343    22356      +13     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@wudidapaopao wudidapaopao changed the title feat: rewrite count over non-nullable columns feat: rewrite count(non-nullable column) to count(1) Oct 3, 2026
@wudidapaopao

Copy link
Copy Markdown
Contributor Author

@neilconway Thanks for the suggestion! I split the COUNT(non-nullable column) -> COUNT(1) rewrite out of #25849 into this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate functions Changes to functions implementation optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rewrite COUNT over non-nullable columns to COUNT(1)

2 participants