Repository navigation

[WIP][POC] Pgaur on pair evaluation - #50791

Draft

prtkgaur wants to merge 14 commits into

apache:mainfrom

prtkgaur:pgaurOnPairEvaluation

Draft

[WIP][POC] Pgaur on pair evaluation#50791
prtkgaur wants to merge 14 commits into
apache:mainfrom
prtkgaur:pgaurOnPairEvaluation

Commits on Oct 4, 2026
- Vendor the FSST library
  
  ArnavBalyan
  authored and
  sfc-gh-pgaur
  committed
  1292ec3
- Add FSST encoding to Parquet
  
  ArnavBalyan
  authored and
  sfc-gh-pgaur
  committed
  0242963
- Implement the OnPair string codec
  sfc-gh-pgaur
  committed
  e1201a3
- Extract common prefixes before string encoding
  sfc-gh-pgaur
  committed
  a0d9726
- Benchmark FSST and OnPair on shared corpora
  sfc-gh-pgaur
  committed
  f2ad37f
- Benchmark cascaded string compression
  sfc-gh-pgaur
  committed
  241e319
- Validate every OnPair code width
  sfc-gh-pgaur
  committed
  206bff9
- Decode OnPair tokens through a fixed-stride view
  sfc-gh-pgaur
  committed
  6b774ef
- Decode packed OnPair codes in periodic groups
  sfc-gh-pgaur
  committed
  1cd4013
- Compare OnPair with a 16-bit FSST table
  sfc-gh-pgaur
  committed
  37bf393
- Build OnPair dictionaries from the observed alphabet
  sfc-gh-pgaur
  committed
  1af7b8d
- Select the OnPair dictionary budget by decode cost
  sfc-gh-pgaur
  committed
  0c5c64c
- Harden OnPair evaluation inputs and buffers
  sfc-gh-pgaur
  committed
  b4183a7
- Build OnPair evaluation targets with CMake
  sfc-gh-pgaur
  committed
  53149d6