Repository navigation
[C++][Parquet] Enable Parquet DB support on Big Endian (IBM Z) systems #48151
Description
Describe the enhancement requested
Apache Arrow repo support is not enabled on IBM Z systems.. There are ~14
test suite failures seen. The list is as below
Parquet-test-failure-summary.txt
test suite failures encountered when tried to build and run the tests on s390x machine.
The test suite failures are listed below:
arrow-acero-asof-join-node-test - Aborted/Core dumped
arrow-ipc-message-internal-test - 1 Test case Failure
arrow-acero-hash-join-node-test - 1 Test case Failure
arrow-ipc-message-internal-test - 1 Test case Failure
arrow-dataset-file-parquet-test - Testcase Hang
arrow-bit-utility-test - 1 Test case Failure
arrow-compute-row-test - Aborted/Core dumped
parquet-arrow-internals-test - Testcase Hang
parquet-arrow-reader-writer-test - Aborted/Core dumped
parquet-chunker-test - Aborted/Core dumped
parquet-encoding-test - 19 Test case Failures
parquet-internals-test - 43 Test case Failures
parquet-reader-test - Aborted/Core dumped
parquet-writer-test - 5 Test case Failures
Couple of testcase execution hangs as well. The error details are attached in the txt file.
In general, there no big-endian support enabled for this repo.
Component(s)
Parquet
Activity
Vishwanatha-HD commentedon Nov 17, 2025
I want to assign this issue to myself
"take"
Vishwanatha-HD commentedon Nov 17, 2025
I have worked on resolving all these issue and tested it on the IBM Z (s390x) platforms. The fix is working fine and all the test suites are passing. I will raise a Github PR to fix the above issues.
kou commentedon Nov 18, 2025
Thanks.
If your PR is large, we may ask you to split your PR to some small PRs.
[-][C++][Parquet][Arrow] Enable Parquet DB support on Big Endian (IBM Z) systems for Apache Arrow repo[/-][+][C++][Parquet] Enable Parquet DB support on Big Endian (IBM Z) systems[/+]Vishwanatha-HD commentedon Nov 18, 2025
Hi @kou..
Thanks for your comments. Yes, my changes are medium to large. As you are suggesting, I will split it into few smaller PRs and raise all of them for review.. thanks !!
apacheGH-48151: [C++][Parquet] Fix arrow-ipc-message-internal-test & …
Vishwanatha-HD commentedon Nov 18, 2025
Hi @kou
I have raised a first PR which fixes two of the test case failures. I will raise further PRs which has fixes to the remaining testcase failures in sometime. Thanks
kou commentedon Nov 19, 2025
Thanks for splitting to smaller PRs.
Could you open sub-issues for smaller PRs to track them?
Vishwanatha-HD commentedon Nov 19, 2025
Hi @kou..
I will try my best to split my fixes into smaller PRs and also create sub-issues for them.
But please note that, since most of the fixes are related to endianness issues and since most of the fixes have common codd path involved in it for eg: plain encoder & decoders, column reader & writers etc., I wont be able to breakdown and create issues for each one of the failing testcases mentioned above..
Hence, I am trying to separate out the testcase fixes which are isolated in nature and create separate PRs for them.. For rest ohter fixes, which have common files involved in it, I will have to combine all of them into one and raise a PR..
Otherwise, I will end-up breaking overall build.. Hope this is fine with you.. Thanks..
apacheGH-48151: [C++][Parquet] Fix arrow-acero-asof-join-node-test fa…
24 remaining items
apacheGH-48151: [C++][Parquet] Fix arrow-ipc-message-internal-test & …
apacheGH-48151: [C++][Parquet] Fix arrow-acero-asof-join-node-test fa…
apacheGH-48151: [C++][Parquet] Fix arrow-acero-asof-join-node-test fa…
apacheGH-48151: [C++][Parquet] Fix arrow-ipc-message-internal-test & …
apacheGH-48151: [C++][Parquet] Fix arrow-ipc-message-internal-test & …
apacheGH-48151: [C++][Parquet] Fix arrow-ipc-message-internal-test & …
apacheGH-48151: [C++][Parquet] Fix arrow-ipc-message-internal-test & …
apacheGH-48151: [C++][Parquet] Fix arrow-ipc-message-internal-test fa…
GH-48176: [C++][Parquet] Fix arrow-ipc-message-internal-test failure (#…
apacheGH-48151: [C++][Parquet] Fix arrow-acero-asof-join-node-test fa…
apacheGH-48151: [C++][Parquet] Fix arrow-acero-asof-join-node-test fa…
apacheGH-48218: [C++][Parquet] Fix Util & Level Conversion logic on b…
apacheGH-48176: [C++][Parquet] Fix arrow-ipc-message-internal-test fa…
apacheGH-48151: [C++][Parquet] Fix arrow-acero-asof-join-node-test fa…
apacheGH-48151: [C++][Parquet] Fix arrow-acero-asof-join-node-test fa…