Reuse aggregate-derived join domains as detail-side payloads - #23113
Draft
pentschev wants to merge 2 commits into
Draft
Reuse aggregate-derived join domains as detail-side payloads#23113pentschev wants to merge 2 commits into
pentschev wants to merge 2 commits into
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Replace eligible detail-side join inputs with an existing aggregate-derived payload in join filter pushdown. This keeps the upstream semi-domain filtering from current main while removing the remaining raw detail scan from Q18-style plans, reducing memory pressure and improving runtime without depending on the separate reused semi-domain candidate branch.
pentschev
force-pushed
the
cudf-polars/join-aggregate-domain-reuse
branch
from
August 10, 2026 20:39
01e286b to
fbc8ea5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This extends
join_filter_pushdownto recognize when an aggregate-derived join domain can be reused as the payload for a later detail-side join input.The motivating case is PDSH Q18. The query computes the set of
l_orderkeyvalues whosesum(l_quantity) > 300, and current planning can already use that result as a semi-join domain to filterorders. However, the finalordersxlineitemjoin still reads the rawlineitemdetail side again.With this change, when the aggregate-derived domain is known to contain the join key and the payload columns needed by the later join, the planner can reuse that derived domain to feed the detail side as well. The original full join is still preserved, so the rewrite is a row-reduction optimization rather than a semantic shortcut.
Q18 plan before, relevant subtree
Q18 plan after, relevant subtree
Materially, this reduces Q18 memory pressure and improves runtime by avoiding the final unfiltered detail-side
lineiteminput. In the latest standalone validation, the full SF30K Q1-Q22 workflow completed successfully on 6xNVL4 nodes with validation passing in 146s lukewarm/128s hot.