Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
03ef8ad
Add dynamic join key prefilter planning
pentschev Jun 25, 2026
6b71a95
Add generic derived join-domain prefilters
pentschev Jun 25, 2026
b22929f
Add profitability guards for derived join prefilters
pentschev Jun 25, 2026
09f0474
Reject boolean join prefilter key limits
pentschev Jun 26, 2026
6f95d33
Consolidate optional config converters
pentschev Jun 26, 2026
519cd1d
Remove legacy bloom prefilter threshold path
pentschev Jun 26, 2026
d1cd0a1
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-pr…
pentschev Jun 26, 2026
09a744e
Fix linting
pentschev Jun 26, 2026
7da769c
Accept integer join prefilter thresholds
pentschev Jun 26, 2026
fd6ba23
Implement default in-place without additional variable
pentschev Jun 26, 2026
fbe886b
Document join prefilter key prefix limit
pentschev Jun 26, 2026
f01e62e
Use dataclass conversion for join prefilter trace
pentschev Jun 26, 2026
ffd5e8f
Remove redundant join prefilter decision state
pentschev Jun 26, 2026
6bee038
Remove unused keyless prefilter skip
pentschev Jun 26, 2026
47f2ce4
Assert matching join prefilter key counts
pentschev Jun 26, 2026
97466a3
Track only join prefilter apply side
pentschev Jun 26, 2026
543a73d
Remove unused prefilter partition trace
pentschev Jun 26, 2026
0dfc80d
Clarify dynamic join collective IDs
pentschev Jun 26, 2026
94e132c
Document actor trace extra metadata
pentschev Jun 26, 2026
918417d
Fix docstring linting
pentschev Jun 26, 2026
29b950e
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-pr…
pentschev Jun 26, 2026
c75249d
Fix optional join prefilter default typing
pentschev Jun 26, 2026
42871f0
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-pr…
pentschev Jun 26, 2026
dc1ffb2
Merge branch 'cudf-polars/join-prefilter-core' into cudf-polars/join-…
pentschev Jun 29, 2026
0dafc3e
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-pr…
pentschev Jun 29, 2026
d81c198
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-do…
pentschev Jun 29, 2026
4cc6213
Use column expressions for join domain keys
pentschev Jun 29, 2026
c148219
Use shared DAG utilities for domain prefilter planning
pentschev Jun 29, 2026
fe17507
Restore optional float config conversion
pentschev Jun 29, 2026
1c0e3df
Reanalyze rewritten join-domain subtrees
pentschev Jun 29, 2026
702c6ad
Generalize default factory result typing
pentschev Jun 29, 2026
a37608d
Document join prefilter selection inputs
pentschev Jun 29, 2026
6bc7559
Simplify join prefilter decision flow
pentschev Jun 29, 2026
5ac7e79
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-pr…
pentschev Jun 29, 2026
903084d
Fix missing coverage
pentschev Jun 29, 2026
4538aef
Fix one more missing coverage
pentschev Jun 29, 2026
c55aa6c
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-pr…
pentschev Jun 29, 2026
4a938fc
Merge branch 'cudf-polars/join-prefilter-core' into cudf-polars/join-…
pentschev Jun 29, 2026
688ec65
Merge branch 'cudf-polars/join-domain-prefilter' into cudf-polars/joi…
pentschev Jun 29, 2026
d1936aa
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-pr…
pentschev Jun 29, 2026
eab9726
Merge branch 'cudf-polars/join-prefilter-core' into cudf-polars/join-…
pentschev Jun 29, 2026
93ca013
Merge branch 'cudf-polars/join-domain-prefilter' into cudf-polars/joi…
pentschev Jun 29, 2026
47ac908
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-do…
pentschev Jun 29, 2026
8cf2df7
Merge branch 'cudf-polars/join-domain-prefilter' into cudf-polars/joi…
pentschev Jun 29, 2026
b3049c2
Add missing coverage for JOIN_DOMAIN_PREFILTER_TRACE
pentschev Jun 29, 2026
29e0044
Merge branch 'cudf-polars/join-domain-prefilter' into cudf-polars/joi…
pentschev Jun 29, 2026
1a0d1f5
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-do…
pentschev Jun 30, 2026
0b93f55
Simplify join-domain candidate selection
pentschev Jun 30, 2026
0226a83
Track column bindings through domain subplans
pentschev Jun 30, 2026
8bd9023
Keep domain prefilter replacement side-scoped
pentschev Jun 30, 2026
9511c94
Format join-domain review updates
pentschev Jun 30, 2026
ae443dd
Trace bindings through filtering joins
pentschev Jun 30, 2026
571181b
Check cheap domain guards first
pentschev Jul 1, 2026
cf59505
Inline target prefilter construction
pentschev Jul 1, 2026
3a57f14
Clarify bound key projection
pentschev Jul 1, 2026
4261279
Decouple join-domain prefilter configuration
pentschev Jul 1, 2026
ff32f43
Test nullable join-domain prefilters
pentschev Jul 1, 2026
c21cc13
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-do…
pentschev Jul 1, 2026
011ba87
Validate join-domain prefilter configuration
pentschev Jul 1, 2026
a5041ec
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-do…
pentschev Jul 2, 2026
7188a61
Use optional join-domain prefilter options
pentschev Jul 2, 2026
eabbdba
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-do…
pentschev Jul 2, 2026
9cf3029
Fix join-domain prefilter CI regressions
pentschev Jul 2, 2026
c07ca0b
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-do…
pentschev Jul 2, 2026
fbc56ec
Merge remote-tracking branch 'upstream/main' into cudf-polars/join-do…
pentschev Jul 3, 2026
2ab4f10
Merge branch 'cudf-polars/join-domain-prefilter' into cudf-polars/joi…
pentschev Jul 3, 2026
7f9d2b2
Reuse existing semi-join domains
pentschev Jul 3, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/cudf/source/cudf_polars/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ Most users interact with them through `StreamingOptions` fields rather than dire
.. automodule:: cudf_polars.utils.config
:members:
DynamicPlanningOptions,
JoinDomainPrefilterOptions,
MemoryResourceConfig,
ParquetOptions,
StreamingExecutor,
Expand Down
1 change: 1 addition & 0 deletions docs/cudf/source/cudf_polars/options.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ Environment variables follow these patterns:
| `broadcast_limit` | Maximum number of bytes for broadcast joins. | auto |
| `target_partition_size` | Target partition size in bytes. Used for IO and dynamic planning. `0` means auto. | auto |
| `dynamic_planning` | Dynamic planning configuration, dict or {class}`~cudf_polars.utils.config.DynamicPlanningOptions`. `None` disables. | enabled |
| `join_domain_prefilter` | Join-domain prefilter configuration, dict or {class}`~cudf_polars.utils.config.JoinDomainPrefilterOptions`. `None` disables. | enabled |
| `sink_to_directory` | Whether `.sink_*()` writes its output as a directory. The `spmd`, `ray`, and `dask` engines always use `True`; passing `False` raises `ValueError`. | `True` |

### Category: `engine`
Expand Down
12 changes: 12 additions & 0 deletions python/cudf_polars/cudf_polars/engine/options.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@

from cudf_polars.utils.config import (
DynamicPlanningOptions,
JoinDomainPrefilterOptions,
ParquetOptions,
)

Expand Down Expand Up @@ -247,6 +248,14 @@ class StreamingOptions:
Env: ``CUDF_POLARS__EXECUTOR__DYNAMIC_PLANNING``.
Default: enabled.
Category: executor.
join_domain_prefilter
Join-domain prefilter config, dict or
:class:`~cudf_polars.utils.config.JoinDomainPrefilterOptions`. ``None``
disables the rewrite.
Env: ``CUDF_POLARS__EXECUTOR__JOIN_DOMAIN_PREFILTER`` and
``CUDF_POLARS__EXECUTOR__JOIN_DOMAIN_PREFILTER__*``.
Default: enabled.
Category: executor.
sink_to_directory
Whether multi-partition sink operations should write to a directory
rather than a single file. The ``spmd``/``ray``/``dask`` engines
Expand Down Expand Up @@ -341,6 +350,9 @@ class StreamingOptions:
dynamic_planning: dict[str, Any] | DynamicPlanningOptions | None | Unspecified = (
_opt("executor")
)
join_domain_prefilter: (
dict[str, Any] | JoinDomainPrefilterOptions | None | Unspecified
) = _opt("executor")
sink_to_directory: bool | Unspecified = _opt(
"executor", "CUDF_POLARS__EXECUTOR__SINK_TO_DIRECTORY", parse_boolean
)
Expand Down
Loading
Loading