Skip to content

Return a zero-copy slice from the filter kernel when the predicate selects one contiguous run #11255

Open

Description

@alamb

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

#10859 adds a small special case in narrow_batch in the parquet reader: when the boolean filter selects exactly one contiguous run of rows, the batch is narrowed with a zero-copy RecordBatch::slice instead of filter_record_batch.

fn narrow_batch(batch: &RecordBatch, filter: &BooleanArray, true_count: usize) -> Result<RecordBatch, ArrowError> {
    let mut slices = SlicesIterator::new(filter);
    let (start, end) = slices.next().expect("a partially selected filter has a true slice");
    if end - start == true_count {
        Ok(batch.slice(start, end - start))
    } else {
        filter_record_batch(batch, filter)
    }
}

This optimization only helps that one call site, but could also be applied to the filter kernel in arrow_select.

It already returns a zero-copy slice when the predicate selects every row

Describe the solution you'd like

Teach the arrow_select filter kernel to recognize a single contiguous and combine narrow_batch in the parquet reader to call filter_record_batch.

Describe alternatives you've considered

  • Leave the special case in narrow_batch. This works but benefits only the parquet fused-predicate path and duplicates logic the kernel should own.
  • Have callers check for a single run themselves before calling filter. This pushes the same duplication to every call site.

Additional context

Activity

  1. Abhisheklearn12 commented on Sep 28, 2026

    @Abhisheklearn12
    Contributor

    take

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions