perf(parquet): Batch DELTA decode + SIMD filter pushdown#3
Closed
jaylisde wants to merge 1 commit into
Closed
Conversation
4 tasks
jaylisde
force-pushed
the
pr/delta-bp-perf
branch
from
May 30, 2026 07:14
92e33b7 to
2d17343
Compare
This was referenced Jun 1, 2026
jaylisde
added a commit
that referenced
this pull request
Jun 3, 2026
…sitor paths Builds on #3 (PR A) and #4 (PR B) to close the remaining DELTA-vs-PLAIN/DICT scan gap on TPC-H Q12. 1. Inline SIMD bit-unpack kernel in DeltaBpDecoder::decodeLongs. When the read aligns at a miniblock start and consumes a whole miniblock, dispatch on bit_width 0..32 to a compile-time specialized kernel: - bw 0: arithmetic-sequence fast path (no bit-extract). - bw 1..16: 4-value/iter, single unaligned 64-bit load (4*16 = 64 bits fit in one u64 window). - bw 17..32: 2-value/iter, __uint128_t funnel-shift (lowers to two u64 loads + SHRD on x86_64). The trailing u64 read is safe because intra-page overshoots fall into the next miniblock and the last miniblock has PageReader::kPageReadPadding (8) trailing bytes guaranteed. bit_widths 33..64 fall through to the per-row scalar inner loop, which is unchanged. 2. New readWithVisitorSparseBuffered path for the !Visitor::dense + !hasNulls + deterministic filter + NoHook + integral DataType case (the hot path on Q12 because filter chaining produces sparse row sets after l_shipmode is applied). Decodes kBatch=1024 physical values into a stack buffer via decodeLongs (which now uses the SIMD kernel above), then walks the visitor's sparse rows array using rows[k] - batchPhysStart as buffer index. The existing per-row visitor.process is preserved — only the decode side is batched. n is capped to the visitor's residual physical span so the decoder never advances past what the visitor will consume. 3. readWithVisitorDenseBatched kBatch raised from 256 to 1024 to amortize the chunk-loop overhead now that decodeLongs is much faster per call. 4. Add velox/dwio/parquet/tests/reader/DeltaBpDecoderTest.cpp: - Hand-rolled DeltaEncoder for byte-stream control in tests. - 11 boundary tests: bit_widths 0/8/10/16/24/32, multi-block in a single readValues, mid-miniblock split across two readValues calls, negative minDelta, bit_width > 32 fallback, narrowing to int32_t. - 32 parameterized roundtrip tests covering bit_widths 1..32, each forcing the encoder to pick exactly that width by saturating one residual to (1<<bw)-1. Bench (TPC-H Q12 SF10, DELTA-encoded lineitem, num_drivers=4, 5-run median): wall #3 baseline 897 ms +#4 (PR B) 748 ms +this PR 626 ms That is -16.3% on top of PR B, -30.2% from the PR A baseline. PLAIN/DICT parquet paths are unchanged. Test plan - velox_dwio_parquet_delta_bp_decoder_test: 43 tests pass (11 boundary + 32 parameterized bw 1..32). - velox_dwio_parquet_table_scan_test: 54 tests pass (includes the 7 delta tests added in PR A). - velox_parquet_e2e_filter_test: 34 tests pass. Tracking: #2.
jaylisde
force-pushed
the
pr/delta-bp-perf
branch
from
June 3, 2026 06:30
5e5071f to
a53751d
Compare
jaylisde
added a commit
that referenced
this pull request
Jun 3, 2026
…sitor paths Builds on #3 (PR A) and #4 (PR B) to close the remaining DELTA-vs-PLAIN/DICT scan gap on TPC-H Q12. 1. Inline SIMD bit-unpack kernel in DeltaBpDecoder::decodeLongs. When the read aligns at a miniblock start and consumes a whole miniblock, dispatch on bit_width 0..32 to a compile-time specialized kernel: - bw 0: arithmetic-sequence fast path (no bit-extract). - bw 1..16: 4-value/iter, single unaligned 64-bit load (4*16 = 64 bits fit in one u64 window). - bw 17..32: 2-value/iter, __uint128_t funnel-shift (lowers to two u64 loads + SHRD on x86_64). The trailing u64 read is safe because intra-page overshoots fall into the next miniblock and the last miniblock has PageReader::kPageReadPadding (8) trailing bytes guaranteed. bit_widths 33..64 fall through to the per-row scalar inner loop, which is unchanged. 2. New readWithVisitorSparseBuffered path for the !Visitor::dense + !hasNulls + deterministic filter + NoHook + integral DataType case (the hot path on Q12 because filter chaining produces sparse row sets after l_shipmode is applied). Decodes kBatch=1024 physical values into a stack buffer via decodeLongs (which now uses the SIMD kernel above), then walks the visitor's sparse rows array using rows[k] - batchPhysStart as buffer index. The existing per-row visitor.process is preserved — only the decode side is batched. n is capped to the visitor's residual physical span so the decoder never advances past what the visitor will consume. 3. readWithVisitorDenseBatched kBatch raised from 256 to 1024 to amortize the chunk-loop overhead now that decodeLongs is much faster per call. 4. Add velox/dwio/parquet/tests/reader/DeltaBpDecoderTest.cpp: - Hand-rolled DeltaEncoder for byte-stream control in tests. - 11 boundary tests: bit_widths 0/8/10/16/24/32, multi-block in a single readValues, mid-miniblock split across two readValues calls, negative minDelta, bit_width > 32 fallback, narrowing to int32_t. - 32 parameterized roundtrip tests covering bit_widths 1..32, each forcing the encoder to pick exactly that width by saturating one residual to (1<<bw)-1. Bench (TPC-H Q12 SF10, DELTA-encoded lineitem, num_drivers=4, 5-run median): wall #3 baseline 897 ms +#4 (PR B) 748 ms +this PR 626 ms That is -16.3% on top of PR B, -30.2% from the PR A baseline. PLAIN/DICT parquet paths are unchanged. Test plan - velox_dwio_parquet_delta_bp_decoder_test: 43 tests pass (11 boundary + 32 parameterized bw 1..32). - velox_dwio_parquet_table_scan_test: 54 tests pass (includes the 7 delta tests added in PR A). - velox_parquet_e2e_filter_test: 34 tests pass. Tracking: #2.
Layered optimizations on top of facebookincubator#17633 to close the remaining DELTA-vs-PLAIN/DICT scan gap on TPC-H Q12 (DELTA-encoded lineitem). 1. Batched fast path in DeltaBpDecoder::readWithVisitor — for dense integer reads with deterministic filters and no hook, decode a chunk into the visitor's output buffer and dispatch one visitor.processRun() per chunk. 2. Inline readLong() state hoist in decodeLongs — keeps bufferStart_, the two remaining counters, lastValue_, minDelta_ and deltaBitWidth_ in registers across the inner loop; advances a running bitOffset instead of recomputing valuesConsumed * deltaBitWidth_ per row. 3. New base ColumnVisitor::processRun with two SIMD subpaths via dwio::common::processFixedFilter. Specialized visitors keep their own overrides; this base entry is reachable only from the new DELTA batched path. 4. Fix scalar-tail / SIMD-counter sync in processRun. The dense batched caller commits the SIMD loop's local counter via setNumValues(numValues), but the scalar tail's process() advances the reader's own numValues_ counter, so any rows produced by the tail when numInput < kWidth were silently dropped. Sync the local into the reader before the tail and pull the tail's increments back into the local after, so the final commit covers both phases. 5. Seven boundary tests in ParquetTableScanTest covering bit_width=0 constant deltas, narrow widths, the 32/33-bit boundary, mixed miniblock widths, a wide-and-negative pattern, and the scalar-tail regression above (deltaBinaryPackedFilterScalarTail).
jaylisde
force-pushed
the
pr/delta-bp-perf
branch
from
June 3, 2026 06:49
a53751d to
4dde7ca
Compare
jaylisde
added a commit
that referenced
this pull request
Jun 3, 2026
…sitor paths Builds on #3 (PR A) and #4 (PR B) to close the remaining DELTA-vs-PLAIN/DICT scan gap on TPC-H Q12. 1. Inline SIMD bit-unpack kernel in DeltaBpDecoder::decodeLongs. When the read aligns at a miniblock start and consumes a whole miniblock, dispatch on bit_width 0..32 to a compile-time specialized kernel: - bw 0: arithmetic-sequence fast path (no bit-extract). - bw 1..16: 4-value/iter, single unaligned 64-bit load (4*16 = 64 bits fit in one u64 window). - bw 17..32: 2-value/iter, __uint128_t funnel-shift (lowers to two u64 loads + SHRD on x86_64). The trailing u64 read is safe because intra-page overshoots fall into the next miniblock and the last miniblock has PageReader::kPageReadPadding (8) trailing bytes guaranteed. bit_widths 33..64 fall through to the per-row scalar inner loop, which is unchanged. 2. New readWithVisitorSparseBuffered path for the !Visitor::dense + !hasNulls + deterministic filter + NoHook + integral DataType case (the hot path on Q12 because filter chaining produces sparse row sets after l_shipmode is applied). Decodes kBatch=1024 physical values into a stack buffer via decodeLongs (which now uses the SIMD kernel above), then walks the visitor's sparse rows array using rows[k] - batchPhysStart as buffer index. The existing per-row visitor.process is preserved — only the decode side is batched. n is capped to the visitor's residual physical span so the decoder never advances past what the visitor will consume. 3. readWithVisitorDenseBatched kBatch raised from 256 to 1024 to amortize the chunk-loop overhead now that decodeLongs is much faster per call. 4. Add velox/dwio/parquet/tests/reader/DeltaBpDecoderTest.cpp: - Hand-rolled DeltaEncoder for byte-stream control in tests. - 11 boundary tests: bit_widths 0/8/10/16/24/32, multi-block in a single readValues, mid-miniblock split across two readValues calls, negative minDelta, bit_width > 32 fallback, narrowing to int32_t. - 32 parameterized roundtrip tests covering bit_widths 1..32, each forcing the encoder to pick exactly that width by saturating one residual to (1<<bw)-1. Bench (TPC-H Q12 SF10, DELTA-encoded lineitem, num_drivers=4, 5-run median): wall #3 baseline 897 ms +#4 (PR B) 748 ms +this PR 626 ms That is -16.3% on top of PR B, -30.2% from the PR A baseline. PLAIN/DICT parquet paths are unchanged. Test plan - velox_dwio_parquet_delta_bp_decoder_test: 43 tests pass (11 boundary + 32 parameterized bw 1..32). - velox_dwio_parquet_table_scan_test: 54 tests pass (includes the 7 delta tests added in PR A). - velox_parquet_e2e_filter_test: 34 tests pass. Tracking: #2.
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Layered optimizations on top of facebookincubator#17633 to close the remaining DELTA-vs-PLAIN/DICT gap on TPC-H Q12 (DELTA-encoded lineitem).
DeltaBpDecoder::readWithVisitor— for dense integer reads with deterministic filters and no hook, decode a chunk into the visitor's output buffer and dispatch onevisitor.processRun()per chunk.readLongstate hoist — keepbufferStart_, the two remaining counters,lastValue_,minDelta_anddeltaBitWidth_in registers across the inner loop; advance a runningbitOffsetinstead of recomputingvaluesConsumed * deltaBitWidth_per row.ColumnVisitor::processRunwith two SIMD subpaths viadwio::common::processFixedFilter. Specialized visitors keep their own overrides; this base entry is reachable only from the new DELTA batched path. Includes a scalar-tail / SIMD-counter sync fix (commit5e5071fc) — without it,numInput < kWidthcases let the scalar tail overwrite earlier rows because the surroundingsetNumValues(local)doesn't include the tail's appends. New testdeltaBinaryPackedFilterScalarTailreproduces the bug on the pre-fix variant.ParquetTableScanTestcovering bit_width=0 constant deltas, narrow widths, the 32/33-bit boundary, mixed miniblock widths, a wide-and-negative pattern, and the scalar-tail regression above.Bench
TPC-H Q12 SF10, DELTA-encoded lineitem (3 runs, median):
PLAIN/DICT (default) parquet unchanged within noise (614 → 608 ms scan CPU on Q12 SF10 PLAIN).
Test plan
velox_dwio_parquet_table_scan_test --gtest_filter=*deltaBinaryPacked*velox_dwio_parquet_table_scan_testsuite still greenTracking
Part of #2.