[parquet] DELTA decoder performance follow-up to PR facebookincubator#17633
Background
After PR facebookincubator#17633, DELTA-encoded scan CPU on TPC-H Q12 SF10 lineitem is still ~2.76s vs ~614ms for PLAIN/DICT (~4.5x) — flagged by @mbasmanova in review. This issue tracks the follow-up work to close that gap.
Diagnosis
perf on the post-facebookincubator#17633 hot path:
| Source |
Cost |
Cause |
readLong per-row loop |
~22% |
Member store/load round-trip on every row (totalValuesRemaining_, valuesRemainingCurrentMiniBlock_, lastValue_) |
DeltaByteArrayDecoder ctor per page |
~13% |
make_unique<RleBpDecoder> × 3 + std::string lastValue_ reallocation |
| Filter evaluation |
~30% |
Per-row BigintRange::test() instead of SIMD batch |
Plan
Three sequential PRs, each with its own design write-up:
- A —
DeltaBpDecoder batch decode + SIMD filter pushdown (devirtualizes per-row state, adds processRun SIMD path).
- B —
DeltaByteArrayDecoder rework (devirtualize, lazy-init child decoders, drop per-page heap alloc).
- C — SIMD bit-unpack inside
decodeLongs + new sparse-buffered visitor path for the post-filter case.
Outcome
PLAIN/DICT scan CPU is 614 ms (reference). Post-A+B+C: DELTA scan CPU 1.38 s on TPC-H Q12 SF10 (~2.2x of PLAIN/DICT, down from ~4.5x). Q12 wall 1.02 s → 0.626 s. Detailed bench tables in each PR. PLAIN/DICT path unchanged within noise.
Tracking
The three layers merged into a single PR for upstream review since
they compound for the headline speedup:
[parquet] DELTA decoder performance follow-up to PR facebookincubator#17633
Background
After PR facebookincubator#17633, DELTA-encoded scan CPU on TPC-H Q12 SF10 lineitem is still ~2.76s vs ~614ms for PLAIN/DICT (~4.5x) — flagged by @mbasmanova in review. This issue tracks the follow-up work to close that gap.
Diagnosis
perf on the post-facebookincubator#17633 hot path:
readLongper-row looptotalValuesRemaining_,valuesRemainingCurrentMiniBlock_,lastValue_)DeltaByteArrayDecoderctor per pagemake_unique<RleBpDecoder>× 3 +std::string lastValue_reallocationBigintRange::test()instead of SIMD batchPlan
Three sequential PRs, each with its own design write-up:
DeltaBpDecoderbatch decode + SIMD filter pushdown (devirtualizes per-row state, addsprocessRunSIMD path).DeltaByteArrayDecoderrework (devirtualize, lazy-init child decoders, drop per-page heap alloc).decodeLongs+ new sparse-buffered visitor path for the post-filter case.Outcome
PLAIN/DICT scan CPU is 614 ms (reference). Post-A+B+C: DELTA scan CPU 1.38 s on TPC-H Q12 SF10 (~2.2x of PLAIN/DICT, down from ~4.5x). Q12 wall 1.02 s → 0.626 s. Detailed bench tables in each PR. PLAIN/DICT path unchanged within noise.
Tracking
The three layers merged into a single PR for upstream review since
they compound for the headline speedup:
rework + SIMD bit-unpack)