[C++] Implement basic functionality for new Binary/Utf8View types

PR #37792 added a basic implementation of the new BinaryView and Utf8View (StringView) format additions. This included mostly just the basic objects (Array, DataType, Builder) and IPC support.
For being able to handle input that uses those new formats, there are some additional features we will have to support (eg receiving a dataset, doing some filtering/projection, and then converting to standard string type).

Array equality (included in initial implementation)
- also expose this as the "equal" kernel
Conversion (cast) to/from the standard binary and string types: GH-42247: [C++] Support casting to and from utf8_view/binary_view #43302
Take / filter
- [C++] Support STRING_VIEW and BINARY_VIEW in "array_take" and "array_filter" functions #43010
Unique / dictionary encoding (included in initial implementation)

https://github.com/apache/arrow/pull/43302/files

This Cast is implemented in C++, I don't know what should be added then 🤔

Updated the top post to indicate cast is implemented now.

Hi! I found this issue after creating #44540 above - thanks Joris for organizing/triaging.

I'm wondering if the cast functionality will be made available in pyarrow in a later release? Or we focusing on polishing up the remaining C++ tasks first?

On another note, another missing functionality I might like to see added is a count_unique kernel for Utf8View. (Hopefully not a big lift as the dictionary implementation already exists.)

I'm wondering if the cast functionality will be made available in pyarrow in a later release? Or we focusing on polishing up the remaining C++ tasks first?

Which version are you using here? I'd try 18.0 for this later, and just support cast if it cannot.

I opened #44336 to track the development of compute kernels - should we just roll that into this PR or keep separate?

Either way is fine for me. In addition to the "basic" kernels (cast, take, filter, etc), there are also all the binary or string-specific kernels (binary_../ascii_../utf8_..)

I'm wondering if the cast functionality will be made available in pyarrow in a later release? Or we focusing on polishing up the remaining C++ tasks first?

Which version are you using here? I'd try 18.0 for this later, and just support cast if it cannot.

I could confirm that casting to pa.string() works very well now, on the 18.0 version of pyarrow!

take

Implementation of basic comparison kernels for binary/utf8 views - #49964

P.S. A lot of cross-referencing here, as a lot of issues correlate with one another.

Implementation of take/filter for binary/utf8 views: #50164

Navigation Menu

Repository navigation

[C++] Implement basic functionality for new Binary/Utf8View types #39634

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[C++] Implement basic functionality for new Binary/Utf8View types #39634

Description

Activity

mapleFU commented on Oct 28, 2024

jorisvandenbossche commented on Oct 28, 2024

kevinli1993 commented on Oct 28, 2024

mapleFU commented on Oct 28, 2024

WillAyd commented on Oct 29, 2024

jorisvandenbossche commented on Oct 30, 2024

kevinli1993 commented on Nov 1, 2024

Periecle commented on May 12, 2026

Periecle commented on May 12, 2026

Periecle commented on Jun 12, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions