Skip to content

GH-53138: [R][Parquet] Expose page checksum verification - #53289

Closed
gitedmond wants to merge 3 commits into
apache:mainfrom
gitedmond:gh-53138-r-parquet-checksum
Closed

gitedmond wants to merge 3 commits into
apache:mainfrom
gitedmond:gh-53138-r-parquet-checksum

Conversation

@gitedmond

@gitedmond gitedmond commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Rationale for this change

Fixes #53138. R users cannot enable Parquet page-checksum verification even though C++ already supports it. The corrupt-checksum fixtures therefore read successfully, with no R option to detect the mismatched CRCs.

What changes are included in this PR?

  • Add getter and setter methods to ParquetReaderProperties, forwarding to the existing C++ option, and regenerate the binding registration code.
  • Add regression coverage for the default, enabling and disabling verification, both corrupt-checksum fixtures, valid checksums, and files without checksums.
  • Document the methods and show how to pass the configured properties to read_parquet().

The changes are split into three focused commits: bindings, regression tests, and documentation.

Are these changes tested?

  • Built the unmodified R bindings and reproduced the reported reads: 5,120 and 1,000 rows from the two corrupt-checksum fixtures.
  • Both new test blocks fail against the unmodified bindings and pass after the change (12 assertions). With verification enabled, both fixtures raise CRC checksum verification failed.
  • The complete test-parquet.R selection passes 114 assertions with no failures or errors; two existing CRAN-skipped tests are skipped.
  • The documentation example runs successfully, and rerunning the binding generator produces no further changes.

Validation used R 4.3.3, Arrow R 25.0.1.9000, and Arrow C++ 26.0.0-SNAPSHOT.

Are there any user-facing changes?

R users can opt in with:

reader_props <- ParquetReaderProperties$create()
reader_props$set_page_checksum_verification(TRUE)
read_parquet("data.parquet", reader_props = reader_props)

Verification remains disabled by default. When enabled, existing page checksums are checked; files without page checksums remain readable.

Was AI used for this PR?

Codex assisted with the investigation, implementation, regression tests, documentation, validation, and PR description, and reviewed the changes before submission.

PR code and description written by:

  • Human
  • AI

Reviewed before submission by:

  • Human
  • AI
  • Not reviewed

@github-actions

Copy link
Copy Markdown

Thanks for opening a pull request!

This pull request has been automatically closed because you currently have 4 open pull requests, which is more than the limit of 3.

Due to the increase in pull requests opened by AI bots, and in order to keep the review queue manageable, Apache Arrow limits contributors without repository access to at most 3 concurrently open pull requests. This helps make sure each pull request gets the attention it needs and that work in progress does not go stale.

Once one of your other open pull requests has been merged or closed, you are welcome to reopen this one.

See also:

@github-actions github-actions Bot closed this Oct 12, 2026
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #53138 has been automatically assigned in GitHub to PR creator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[R][Parquet] ParquetReaderProperties does not expose page_checksum_verification, so a corrupt page CRC reads silently

1 participant