[POC] Add BINARY type to cuDF supporting 32-bit and 64-bit offsets

[GregoryKimball]

Description

Many data processing workloads carry binary data as payloads. cuDF has represented this data using a physical type of LIST<UNIT8>, but this representation often hits the limit of 2.1 GB per column due to the size_type limit on list children. If cuDF had a BINARY type, we could naturally represent these kind of payloads, and with the offsetalator pattern from STRINGS we could also support LARGE_BINARY arrow data using 64-bit offsets.

Adding a BINARY type to cuDF brings several benefits:

• enable workloads to carry significant amounts of inline image data without having to carefully partition the table to avoid list overflow
• enable variant data to be represented in columns without list overflow. Status quo is that JSON strings can reach dozens of GB per table, but then the table has to be partitioned before conversion to variant. Variant data uses a physical type of STRUCT<LIST<UNIT8>,LIST<UINT8>> and this could change to STRUCT<BINARY,BINARY>.
• Parquet supports binary data, and currently the cuDF readers and writers have to convert binary to either STRING or LIST<UINT8>. A native binary type would be a more natural target type.
• BINARY type could also let users convert and carry lists of fixed width type, like LIST<FLOAT32> embeddings beyond the 32-bit child limit.

Why haven't we added BINARY yet?

• Adding a new type to cuDF adds another branch to the type dispatcher. Adding a branch to the type dispatcher increases compile time, binary size, and can reduce kernel occupancy.
• LIST<UINT8> has been an adequate workaround
• We were paranoid about adding types because of an occupancy cliff in mixed join and AST. But now we have deprecated the mixed join kernels (for join, chunked probe, post filter instead), and have more mature JIT tools to avoid AST occupancy issues.

What is this POC for?

• measuring the libcudf.so binary size impact of adding a type
  • measurement is promising. Baseline: 188.734 MiB, BINARY branch: 191.350 MiB
• measuring the compilation time impact of adding a type
• measuring the microbenchmark differences from adding a type
  • seems not too bad

• clarifying the blast radius of such a change

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.