Skip to content

Adopt cuco's parametric bloom filter policy in Parquet and streaming - #23049

Merged
rapids-bot[bot] merged 13 commits into
NVIDIA:mainfrom
PointKernel:test-rapids-cmake-1044
Jul 6, 2026
Merged

Adopt cuco's parametric bloom filter policy in Parquet and streaming#23049
rapids-bot[bot] merged 13 commits into
NVIDIA:mainfrom
PointKernel:test-rapids-cmake-1044

Conversation

@PointKernel

@PointKernel PointKernel commented Jun 30, 2026

Copy link
Copy Markdown
Member

Description

This PR migrates the Parquet reader and the streaming bloom filter onto cuco's new parametric_filter_policy (bumped via rapidsai/rapids-cmake#1044). The Parquet reader defines a local arrow_filter_policy alias hashing keys with XXHash_64, and streaming uses the policy directly with identity_hash; no shared cudf policy header is added. Depends on NVIDIA/cuCollections#825, which drops the hasher argument_type requirement so XXHash_64 can be used without an adapter.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@copy-pr-bot

copy-pr-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the CMake CMake build issue label Jun 30, 2026
@PointKernel PointKernel added the DO NOT MERGE Hold off on merging; see PR for details label Jun 30, 2026
@PointKernel

Copy link
Copy Markdown
Member Author

/ok to test 0c16a97

@PointKernel

Copy link
Copy Markdown
Member Author

/ok to test 81828e6

@github-actions github-actions Bot added the libcudf Affects libcudf (C++/CUDA) code. label Jun 30, 2026
@PointKernel

Copy link
Copy Markdown
Member Author

/ok to test ba07b5c

Comment thread cpp/src/io/parquet/arrow_filter_policy.cuh Outdated
@PointKernel

Copy link
Copy Markdown
Member Author

/ok to test 189ac84

@PointKernel

Copy link
Copy Markdown
Member Author

/ok to test 763fb86

@PointKernel PointKernel changed the title Test rapids-cmake PR #1044 Share the Arrow bloom filter policy between Parquet and streaming Jul 1, 2026
@PointKernel
PointKernel marked this pull request as ready for review July 1, 2026 19:53
@PointKernel
PointKernel requested review from a team as code owners July 1, 2026 19:53
@PointKernel
PointKernel requested review from bdice and mattgara July 1, 2026 19:53
@PointKernel PointKernel added non-breaking Non-breaking change improvement Improvement / enhancement to an existing function labels Jul 1, 2026
@PointKernel

Copy link
Copy Markdown
Member Author

This is ready for review and the CMake changes will be reverted once the upstream #1044 gets in.

Comment thread cpp/libcudf_streaming/src/detail/device_bloom_filter.cu Outdated
@PointKernel PointKernel added the 3 - Ready for Review Ready for review by team label Jul 1, 2026

@mhaseeb123 mhaseeb123 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work. LGTM (with test/debug artifacts removed of course)

Comment thread cpp/libcudf_streaming/src/detail/device_bloom_filter.cu Outdated

@wence- wence- left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the caveat around the streaming detail usage.

PointKernel added a commit to NVIDIA/cuCollections that referenced this pull request Jul 2, 2026
Per [NVIDIA/cudf#23049
(comment)](NVIDIA/cudf#23049 (comment)),
`parametric_filter_policy` no longer reads `typename
Hash::argument_type`: `split_hash` is templated on the key type and
deduces everything from the actual call, so hashers without that alias
(e.g. cudf's `XXHash_64`) work directly. `hash_result_type` is fixed to
`uint64_t` with the 64-bit assert moved into `split_hash`.
@PointKernel PointKernel changed the title Share the Arrow bloom filter policy between Parquet and streaming Adopt cuco's parametric bloom filter policy in Parquet and streaming Jul 2, 2026
@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Bloom filter handling now supports a refreshed configuration for Parquet and join-related filtering.
  • Bug Fixes

    • Improved consistency in bloom filter behavior across reading, testing, and join workflows.
    • Updated copyright notices and test setup to align with the latest project configuration.

Walkthrough

Replaces cudf's custom arrow_filter_policy header with cuco::parametric_filter_policy instantiations across the Parquet bloom filter reader, streaming device bloom filter, mark join, and tests. Also removes arrow_filter_policy.cuh entirely and temporarily pins rapids-cmake repo/branch to a test fork for validation.

Changes

Bloom filter policy migration

Layer / File(s) Summary
Arrow filter policy migration in Parquet reader
cpp/src/io/parquet/arrow_filter_policy.cuh, cpp/src/io/parquet/bloom_filter_reader.cu
Deletes the custom arrow_filter_policy header and introduces an in-file alias in bloom_filter_reader.cu using cuco::parametric_filter_policy with XXHash_64, matching Arrow's Block-Split Bloom Filter layout.
Streaming device bloom filter policy update
cpp/libcudf_streaming/src/detail/device_bloom_filter.cu
Updates BloomFilterRefType to use cuco::parametric_filter_policy with cuco::identity_hash<KeyType> instead of the removed arrow_filter_policy.
Mark join bloom filter policy simplification
cpp/src/join/mark_join.cuh
Simplifies bloom_filter_policy_type to cuco::default_filter_policy<hash_value_type>, removing the explicit identity-hash instantiation.
Parquet bloom filter test policy update
cpp/tests/io/parquet_bloom_filter_test.cu
Replaces policy_type with an explicit cuco::parametric_filter_policy<XXHash_64<key_type>, ...> instantiation and updates includes accordingly.
Rapids-cmake test pin
cmake/rapids_config.cmake
Overrides rapids-cmake-repo and rapids-cmake-branch to point to a test fork/branch (cuco-new-filter) for validation.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

  • rapidsai/cudf#22415: Introduced the arrow_filter_policy.cuh header that this PR removes and replaces with cuco::parametric_filter_policy in the same reader and tests.

Suggested labels: cuIO, cuco, feature request

Suggested reviewers: mythrocks, bdice, qbacpey, Matt711, vyasr

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: adopting cuco's parametric bloom filter policy in Parquet and streaming.
Description check ✅ Passed The description matches the changeset and explains the Parquet and streaming bloom filter policy migration.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cmake/rapids_config.cmake`:
- Around line 37-40: Remove the temporary rapids-cmake test pin so main does not
always fetch from the personal fork/branch. The unconditional assignments to
rapids-cmake-repo and rapids-cmake-branch currently bypass the existing NOT
rapids-cmake-branch guard and override caller settings; delete or revert this
test block before merge. Use the rapids-cmake configuration logic in
rapids_config.cmake as the place to restore the normal override behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6ff23f16-b06c-4df9-9ea0-01cde3c29991

📥 Commits

Reviewing files that changed from the base of the PR and between 0732eaa and f7c48a6.

📒 Files selected for processing (6)
  • cmake/rapids_config.cmake
  • cpp/libcudf_streaming/src/detail/device_bloom_filter.cu
  • cpp/src/io/parquet/arrow_filter_policy.cuh
  • cpp/src/io/parquet/bloom_filter_reader.cu
  • cpp/src/join/mark_join.cuh
  • cpp/tests/io/parquet_bloom_filter_test.cu
💤 Files with no reviewable changes (1)
  • cpp/src/io/parquet/arrow_filter_policy.cuh

Comment thread cmake/rapids_config.cmake Outdated
@PointKernel PointKernel added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed DO NOT MERGE Hold off on merging; see PR for details labels Jul 6, 2026
@PointKernel

Copy link
Copy Markdown
Member Author

/merge

@PointKernel PointKernel removed the CMake CMake build issue label Jul 6, 2026
@rapids-bot
rapids-bot Bot merged commit fb23b0d into NVIDIA:main Jul 6, 2026
139 checks passed
@PointKernel
PointKernel deleted the test-rapids-cmake-1044 branch July 6, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3 - Ready for Review Ready for review by team 5 - Ready to Merge Testing and reviews complete, ready to merge improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants