Skip to content

feature(physical-plan): add NullAwareMode to SortMergeJoinExec - #26090

Draft
TinyMurky wants to merge 4 commits into
apache:mainfrom
TinyMurky:support-null-aware-SortMergeJoinExec
Draft

TinyMurky wants to merge 4 commits into
apache:mainfrom
TinyMurky:support-null-aware-SortMergeJoinExec

Conversation

@TinyMurky

Copy link
Copy Markdown
Contributor

This is a draft PR, I'm still working on the issue

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

What is the testing strategy for this PR?

Are there any user-facing changes?

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.71%. Comparing base (6f3f7b7) to head (bd3306c).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...on/physical-plan/src/joins/sort_merge_join/exec.rs 0.00% 22 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #26090      +/-   ##
==========================================
- Coverage   82.72%   82.71%   -0.01%     
==========================================
  Files        1147     1147              
  Lines      448179   448201      +22     
  Branches   448179   448201      +22     
==========================================
- Hits       370751   370744       -7     
- Misses      54898    54923      +25     
- Partials    22530    22534       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@TinyMurky
TinyMurky force-pushed the support-null-aware-SortMergeJoinExec branch from bd3306c to 1b3711b Compare October 9, 2026 15:54
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details
     Cloning apache/main
    Building datafusion-physical-plan v55.1.0 (current)
       Built [  36.250s] (current)
     Parsing datafusion-physical-plan v55.1.0 (current)
      Parsed [   0.145s] (current)
    Building datafusion-physical-plan v55.1.0 (baseline)
       Built [  27.706s] (baseline)
     Parsing datafusion-physical-plan v55.1.0 (baseline)
      Parsed [   0.140s] (baseline)
    Checking datafusion-physical-plan v55.1.0 -> v55.1.0 (no change; assume patch)
     Checked [   0.958s] 223 checks: 222 pass, 1 fail, 0 warn, 31 skip

--- failure method_parameter_count_changed: pub method parameter count changed ---

Description:
A publicly-visible method now takes a different number of parameters, not counting the receiver (self) parameter.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/method_parameter_count_changed.ron

Failed in:
  datafusion_physical_plan::joins::SortMergeJoinExec::try_new takes 7 parameters in /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/ad995f4dd276695f3b49af03f875b86fea22de77/datafusion/physical-plan/src/joins/sort_merge_join/exec.rs:148, but now takes 8 parameters in /home/runner/work/datafusion/datafusion/datafusion/physical-plan/src/joins/sort_merge_join/exec.rs:217

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [  67.380s] datafusion-physical-plan
    Building datafusion-proto-models v55.1.0 (current)
       Built [  18.543s] (current)
     Parsing datafusion-proto-models v55.1.0 (current)
      Parsed [   0.103s] (current)
    Building datafusion-proto-models v55.1.0 (baseline)
       Built [  18.124s] (baseline)
     Parsing datafusion-proto-models v55.1.0 (baseline)
      Parsed [   0.105s] (baseline)
    Checking datafusion-proto-models v55.1.0 -> v55.1.0 (no change; assume patch)
     Checked [   2.613s] 223 checks: 222 pass, 1 fail, 0 warn, 31 skip

--- failure constructible_struct_adds_field: struct exhaustively constructible through public API adds field ---

Description:
A pub struct that could be exhaustively constructed with a literal using only public API has a new pub field, breaking existing exhaustive literals.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/constructible_struct_adds_field.ron

Failed in:
  field SortMergeJoinExecNode.null_aware in /home/runner/work/datafusion/datafusion/datafusion/proto-models/src/generated/prost.rs:2708
  field SortMergeJoinExecNode.null_aware in /home/runner/work/datafusion/datafusion/datafusion/proto-models/src/generated/prost.rs:2708

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [  41.132s] datafusion-proto-models

@github-actions github-actions Bot added the auto detected api change Auto detected API change label Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto detected api change Auto detected API change physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support null-aware (NOT IN) anti / mark joins in SortMergeJoinExec

2 participants