Skip to content

perf(parquet): Reuse DeltaByteArrayDecoder via reset() + devirtualize#4

Closed
jaylisde wants to merge 1 commit into
pr/delta-bp-perffrom
pr/delta-bytearray-perf
Closed

perf(parquet): Reuse DeltaByteArrayDecoder via reset() + devirtualize#4
jaylisde wants to merge 1 commit into
pr/delta-bp-perffrom
pr/delta-bytearray-perf

Conversation

@jaylisde

@jaylisde jaylisde commented May 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

Three layered optimizations on the DELTA_BYTE_ARRAY hot path. Stacks on top of #3 (PR A); the base of this PR is pr/delta-bp-perf.

  1. Devirtualize. Drop DeltaByteArrayDecoderBase. Both DeltaByteArrayDecoder and DeltaLengthByteArrayDecoder are concrete classes; PageReader holds the right pointer type for each encoding. Removes vtable dispatch from per-row readString().
  2. Reuse decoders across pages. Lazy-init child RleBpDecoders via std::optional plus a new reset(pageData_) method instead of make_unique<RleBpDecoder>() per page. Saves three heap allocations per page boundary.
  3. Drop std::string lastValue_. Replace with std::vector<char> + length. Profiling showed ~6.5% in std::string::_M_replace because the prefix-share encoding repeatedly mutates the previous-value buffer inside readString(). The vector path uses raw resize+memcpy with no allocator interaction once the buffer reaches max suffix size.

Bench

TPC-H Q12 SF10, on top of PR A:

Wall Scan CPU
PR A only 0.85 s 2.13 s
+ reset() decoder reuse 0.82 s 2.03 s
+ raw char buffer 0.78 s 1.91 s
+ devirtualize readString 0.74 s 1.74 s

End-to-end (PR A + PR B vs facebookincubator#17633 baseline):

  • Q12 SF10: 1.02 s → 0.74 s wall (-27%); 2.67 s → 1.74 s scan CPU (-35%)
  • Q12 SF100: 16.3 s → 13.1 s wall (-20%); 47.0 s → 33.8 s scan CPU (-28%)

Test plan

  • Existing E2EFilterTest.stringDeltaByteArray (covers prefix+suffix) passes (2.8 s)
  • Existing E2EFilterTest.stringDeltaLengthByteArray (covers length-only encoding, no prefix) passes (1.9 s)
  • Full velox_parquet_e2e_filter_test 34/34 green
  • TPC-H Q12 SF10 wall: 0.85 s (PR A only) → 0.74 s (this PR stacked)

Tracking

Part of #2. Stacked on #3.

@jaylisde
jaylisde force-pushed the pr/delta-bp-perf branch from 92e33b7 to 2d17343 Compare May 30, 2026 07:14
@jaylisde
jaylisde force-pushed the pr/delta-bytearray-perf branch from 9fa0405 to 38359d6 Compare May 30, 2026 07:15
This was referenced Jun 1, 2026
Closed
Open
@jaylisde
jaylisde force-pushed the pr/delta-bytearray-perf branch from 38359d6 to 7967878 Compare June 3, 2026 00:01
jaylisde added a commit that referenced this pull request Jun 3, 2026
@jaylisde
perf(parquet): SIMD bit-unpack for DeltaBpDecoder dense and sparse vi…
18a769a 
…sitor paths

Builds on #3 (PR A) and #4 (PR B) to close
the remaining DELTA-vs-PLAIN/DICT scan gap on TPC-H Q12.

1. Inline SIMD bit-unpack kernel in DeltaBpDecoder::decodeLongs.
   When the read aligns at a miniblock start and consumes a whole
   miniblock, dispatch on bit_width 0..32 to a compile-time
   specialized kernel:
   - bw 0: arithmetic-sequence fast path (no bit-extract).
   - bw 1..16: 4-value/iter, single unaligned 64-bit load (4*16 = 64
     bits fit in one u64 window).
   - bw 17..32: 2-value/iter, __uint128_t funnel-shift (lowers to
     two u64 loads + SHRD on x86_64). The trailing u64 read is safe
     because intra-page overshoots fall into the next miniblock and
     the last miniblock has PageReader::kPageReadPadding (8) trailing
     bytes guaranteed.
   bit_widths 33..64 fall through to the per-row scalar inner loop,
   which is unchanged.

2. New readWithVisitorSparseBuffered path for the
     !Visitor::dense + !hasNulls + deterministic filter +
     NoHook + integral DataType
   case (the hot path on Q12 because filter chaining produces sparse
   row sets after l_shipmode is applied). Decodes kBatch=1024
   physical values into a stack buffer via decodeLongs (which now
   uses the SIMD kernel above), then walks the visitor's sparse rows
   array using rows[k] - batchPhysStart as buffer index. The
   existing per-row visitor.process is preserved — only the decode
   side is batched. n is capped to the visitor's residual physical
   span so the decoder never advances past what the visitor will
   consume.

3. readWithVisitorDenseBatched kBatch raised from 256 to 1024 to
   amortize the chunk-loop overhead now that decodeLongs is much
   faster per call.

4. Add velox/dwio/parquet/tests/reader/DeltaBpDecoderTest.cpp:
   - Hand-rolled DeltaEncoder for byte-stream control in tests.
   - 11 boundary tests: bit_widths 0/8/10/16/24/32, multi-block in a
     single readValues, mid-miniblock split across two readValues
     calls, negative minDelta, bit_width > 32 fallback, narrowing to
     int32_t.
   - 32 parameterized roundtrip tests covering bit_widths 1..32, each
     forcing the encoder to pick exactly that width by saturating one
     residual to (1<<bw)-1.

Bench (TPC-H Q12 SF10, DELTA-encoded lineitem, num_drivers=4, 5-run
median):

                                    wall
  #3 baseline         897 ms
  +#4 (PR B)          748 ms
  +this PR                          626 ms

That is -16.3% on top of PR B, -30.2% from the PR A baseline. PLAIN/DICT
parquet paths are unchanged.

Test plan
- velox_dwio_parquet_delta_bp_decoder_test: 43 tests pass (11 boundary
  + 32 parameterized bw 1..32).
- velox_dwio_parquet_table_scan_test: 54 tests pass (includes the 7
  delta tests added in PR A).
- velox_parquet_e2e_filter_test: 34 tests pass.

Tracking: #2.
@jaylisde
jaylisde force-pushed the pr/delta-bp-perf branch from 5e5071f to a53751d Compare June 3, 2026 06:30
@jaylisde
jaylisde force-pushed the pr/delta-bytearray-perf branch from 7967878 to e987d8d Compare June 3, 2026 06:30
jaylisde added a commit that referenced this pull request Jun 3, 2026
@jaylisde
perf(parquet): SIMD bit-unpack for DeltaBpDecoder dense and sparse vi…
2312b0a 
…sitor paths

Builds on #3 (PR A) and #4 (PR B) to close
the remaining DELTA-vs-PLAIN/DICT scan gap on TPC-H Q12.

1. Inline SIMD bit-unpack kernel in DeltaBpDecoder::decodeLongs.
   When the read aligns at a miniblock start and consumes a whole
   miniblock, dispatch on bit_width 0..32 to a compile-time
   specialized kernel:
   - bw 0: arithmetic-sequence fast path (no bit-extract).
   - bw 1..16: 4-value/iter, single unaligned 64-bit load (4*16 = 64
     bits fit in one u64 window).
   - bw 17..32: 2-value/iter, __uint128_t funnel-shift (lowers to
     two u64 loads + SHRD on x86_64). The trailing u64 read is safe
     because intra-page overshoots fall into the next miniblock and
     the last miniblock has PageReader::kPageReadPadding (8) trailing
     bytes guaranteed.
   bit_widths 33..64 fall through to the per-row scalar inner loop,
   which is unchanged.

2. New readWithVisitorSparseBuffered path for the
     !Visitor::dense + !hasNulls + deterministic filter +
     NoHook + integral DataType
   case (the hot path on Q12 because filter chaining produces sparse
   row sets after l_shipmode is applied). Decodes kBatch=1024
   physical values into a stack buffer via decodeLongs (which now
   uses the SIMD kernel above), then walks the visitor's sparse rows
   array using rows[k] - batchPhysStart as buffer index. The
   existing per-row visitor.process is preserved — only the decode
   side is batched. n is capped to the visitor's residual physical
   span so the decoder never advances past what the visitor will
   consume.

3. readWithVisitorDenseBatched kBatch raised from 256 to 1024 to
   amortize the chunk-loop overhead now that decodeLongs is much
   faster per call.

4. Add velox/dwio/parquet/tests/reader/DeltaBpDecoderTest.cpp:
   - Hand-rolled DeltaEncoder for byte-stream control in tests.
   - 11 boundary tests: bit_widths 0/8/10/16/24/32, multi-block in a
     single readValues, mid-miniblock split across two readValues
     calls, negative minDelta, bit_width > 32 fallback, narrowing to
     int32_t.
   - 32 parameterized roundtrip tests covering bit_widths 1..32, each
     forcing the encoder to pick exactly that width by saturating one
     residual to (1<<bw)-1.

Bench (TPC-H Q12 SF10, DELTA-encoded lineitem, num_drivers=4, 5-run
median):

                                    wall
  #3 baseline         897 ms
  +#4 (PR B)          748 ms
  +this PR                          626 ms

That is -16.3% on top of PR B, -30.2% from the PR A baseline. PLAIN/DICT
parquet paths are unchanged.

Test plan
- velox_dwio_parquet_delta_bp_decoder_test: 43 tests pass (11 boundary
  + 32 parameterized bw 1..32).
- velox_dwio_parquet_table_scan_test: 54 tests pass (includes the 7
  delta tests added in PR A).
- velox_parquet_e2e_filter_test: 34 tests pass.

Tracking: #2.
@jaylisde
jaylisde force-pushed the pr/delta-bp-perf branch from a53751d to 4dde7ca Compare June 3, 2026 06:49
@jaylisde
perf(parquet): Reuse DeltaByteArrayDecoder via reset() + devirtualize
a9359a7 
Three layered optimizations on the DELTA_BYTE_ARRAY hot path:

1. Drop the DeltaByteArrayDecoderBase virtual base. Both
   DeltaByteArrayDecoder and DeltaLengthByteArrayDecoder are concrete
   classes now; PageReader holds the right pointer type for each
   encoding. Removes vtable dispatch from per-row readString().

2. Lazy-init child decoders via std::optional + reset(pageData_).
   Previously every page boundary called make_unique on three
   RleBpDecoders + reset their state; now the decoders are constructed
   once per column-chunk and re-pointed at the next page via reset().
   Saves ~3 heap allocations per page.

3. Replace `std::string lastValue_` with `std::vector<char> lastValueBuf_`
   plus an int32 length. perf showed ~6.5% in std::string::_M_replace
   on every readString() because the prefix-share encoding repeatedly
   resizes the previous-value buffer. The vector path uses raw
   resize+memcpy with no allocator interaction once the buffer reaches
   max suffix size.

Bench (TPC-H Q12 SF10, on top of #pr-A):

                            wall      scan_cpu
  PR A only                 0.85 s    2.13 s
  + reset() reuse           0.82 s    2.03 s
  + raw char buffer         0.78 s    1.91 s
  + devirtualize readString 0.74 s    1.74 s

End-to-end: PR A + PR B brings Q12 SF10 from PR facebookincubator#17633's 1.02 s wall
down to 0.74 s wall (-27%). Q12 SF100 16.3 s -> 13.1 s (-20%).

Tracking: #2
@jaylisde
jaylisde force-pushed the pr/delta-bytearray-perf branch from e987d8d to a9359a7 Compare June 3, 2026 06:49
jaylisde added a commit that referenced this pull request Jun 3, 2026
@jaylisde
perf(parquet): SIMD bit-unpack for DeltaBpDecoder dense and sparse vi…
c01426e 
…sitor paths

Builds on #3 (PR A) and #4 (PR B) to close
the remaining DELTA-vs-PLAIN/DICT scan gap on TPC-H Q12.

1. Inline SIMD bit-unpack kernel in DeltaBpDecoder::decodeLongs.
   When the read aligns at a miniblock start and consumes a whole
   miniblock, dispatch on bit_width 0..32 to a compile-time
   specialized kernel:
   - bw 0: arithmetic-sequence fast path (no bit-extract).
   - bw 1..16: 4-value/iter, single unaligned 64-bit load (4*16 = 64
     bits fit in one u64 window).
   - bw 17..32: 2-value/iter, __uint128_t funnel-shift (lowers to
     two u64 loads + SHRD on x86_64). The trailing u64 read is safe
     because intra-page overshoots fall into the next miniblock and
     the last miniblock has PageReader::kPageReadPadding (8) trailing
     bytes guaranteed.
   bit_widths 33..64 fall through to the per-row scalar inner loop,
   which is unchanged.

2. New readWithVisitorSparseBuffered path for the
     !Visitor::dense + !hasNulls + deterministic filter +
     NoHook + integral DataType
   case (the hot path on Q12 because filter chaining produces sparse
   row sets after l_shipmode is applied). Decodes kBatch=1024
   physical values into a stack buffer via decodeLongs (which now
   uses the SIMD kernel above), then walks the visitor's sparse rows
   array using rows[k] - batchPhysStart as buffer index. The
   existing per-row visitor.process is preserved — only the decode
   side is batched. n is capped to the visitor's residual physical
   span so the decoder never advances past what the visitor will
   consume.

3. readWithVisitorDenseBatched kBatch raised from 256 to 1024 to
   amortize the chunk-loop overhead now that decodeLongs is much
   faster per call.

4. Add velox/dwio/parquet/tests/reader/DeltaBpDecoderTest.cpp:
   - Hand-rolled DeltaEncoder for byte-stream control in tests.
   - 11 boundary tests: bit_widths 0/8/10/16/24/32, multi-block in a
     single readValues, mid-miniblock split across two readValues
     calls, negative minDelta, bit_width > 32 fallback, narrowing to
     int32_t.
   - 32 parameterized roundtrip tests covering bit_widths 1..32, each
     forcing the encoder to pick exactly that width by saturating one
     residual to (1<<bw)-1.

Bench (TPC-H Q12 SF10, DELTA-encoded lineitem, num_drivers=4, 5-run
median):

                                    wall
  #3 baseline         897 ms
  +#4 (PR B)          748 ms
  +this PR                          626 ms

That is -16.3% on top of PR B, -30.2% from the PR A baseline. PLAIN/DICT
parquet paths are unchanged.

Test plan
- velox_dwio_parquet_delta_bp_decoder_test: 43 tests pass (11 boundary
  + 32 parameterized bw 1..32).
- velox_dwio_parquet_table_scan_test: 54 tests pass (includes the 7
  delta tests added in PR A).
- velox_parquet_e2e_filter_test: 34 tests pass.

Tracking: #2.
@jaylisde jaylisde mentioned this pull request Jun 3, 2026
Open
@jaylisde

jaylisde commented Jun 3, 2026

Copy link
Copy Markdown
Owner Author

Squashed into combined PR #6 (perf/delta-decoder-overhaul). Closing this stacked sub-PR — review continues on #6.

@jaylisde jaylisde closed this Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jaylisde