Skip to content

fix: return Int64 from null_count on SQL backends, and 0 on empty frames - #4030

Draft
FBruzzesi wants to merge 2 commits into
mainfrom
fix/duckdb-count-null
Draft

FBruzzesi wants to merge 2 commits into
mainfrom
fix/duckdb-count-null

Conversation

@FBruzzesi

Copy link
Copy Markdown
Member

Description

DuckDB widens integer sum/count_if to HUGEINT, so null_count returned Int128/Decimal(38, 0) on DuckDB and SQLFrame (where collect_schema() also raised), and NULL instead of 0 on empty frames on DuckDB, SQLFrame and Ibis.
A single SQLExpr.null_count (coalesced sum, cast to Int64 outside any over) replaces the three backend implementations, with no slowdown in select, group_by or over, and about half the memory when collecting over results.

What type of PR is this? (check all applicable)

  • 💾 Refactor
  • ✨ Feature
  • 🐛 Bug Fix
  • 🔧 Optimization
  • 📝 Documentation
  • ✅ Test
  • 🐳 Other

Related issues

@FBruzzesi FBruzzesi added fix duckdb Issue is related to duckdb backend spark-like ibis Issue is related to ibis backend labels Oct 5, 2026
@coderabbitai

coderabbitai Bot commented Oct 5, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

duckdb Issue is related to duckdb backend fix ibis Issue is related to ibis backend spark-like

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: null_count() is not an integer dtype on the DuckDB backend, unlike count() and len()

1 participant