Skip to content

[C++] Substrait: add mappings for starts_with, ends_with and match_substring #50988

Description

@jonasdedden

starts_with, ends_with and match_substring have no Substrait mapping, so expressions using them cannot be serialized.

import pyarrow as pa
import pyarrow.compute as pc
from pyarrow.substrait import serialize_expressions

schema = pa.schema([pa.field("cat", pa.string())])
serialize_expressions([pc.starts_with(pc.field("cat"), "al")], ["f"], schema)
pyarrow.lib.ArrowNotImplementedError: No conversion function exists to convert
the Arrow function starts_with to a Substrait call

Same for pc.ends_with and pc.match_substring. Comparisons, isin and arithmetic all serialize fine, so this looks like a per-function gap rather than a structural one.

Substrait defines starts_with, ends_with and contains in functions_string.yaml, which cover all three.

The signatures differ: the Substrait functions take the pattern as a second argument, while the Arrow kernels are unary and carry it in MatchSubstringOptions. So the mapping needs the second argument to be a string literal. Substrait's case_sensitivity option maps onto ignore_case, except for CASE_INSENSITIVE_ASCII which has no Arrow equivalent.

This matters for engines that ingest a PyArrow filter through Substrait, where an unmappable function turns into a hard failure rather than a fallback.

Component(s)

C++, Python

Activity

  1. added a commit that references this issue on Sep 28, 2026
  2. added this to the 26.0.0 milestone on Sep 28, 2026
  3. kou commented on Sep 28, 2026

    @kou
    Member

    Issue resolved by pull request 50989
    #50989

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions