Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
711c864
Implement benchmarks and tests for SearchSorted functionality
Alex-PLACET Apr 2, 2026
35af331
Refactor vector_search_sorted kernel to use ArrayData and add benchma…
Alex-PLACET Apr 2, 2026
a6dc79d
Enhance documentation for search_sorted kernel with detailed implemen…
Alex-PLACET Apr 2, 2026
3540851
Refactor vector_search_sorted kernel to improve null handling and uti…
Alex-PLACET Apr 3, 2026
896296d
Refactor search_sorted kernel: improve error messages and add compreh…
Alex-PLACET Apr 7, 2026
7276b0a
Formatting
Alex-PLACET Apr 8, 2026
dca4c85
Refactor vector_search_sorted kernel: enhance readability and add noe…
Alex-PLACET Apr 8, 2026
48e4d5a
Refactor search_sorted documentation: adjust indentation for clarity
Alex-PLACET Apr 8, 2026
f333968
Implement chunked array support in search_sorted: enhance validation …
Alex-PLACET Apr 22, 2026
ad61b78
Refactor CheckSimpleScalarSearchSorted: support multiple needles and …
Alex-PLACET Apr 22, 2026
3a4f48c
Add tests for chunked run-end encoded values and needles in SearchSorted
Alex-PLACET Apr 22, 2026
0d4ad77
formatting
Alex-PLACET Apr 23, 2026
1670ff3
Refactor search_sorted: extract null counting logic into reusable fun…
Alex-PLACET Apr 23, 2026
c5c3b70
Formatting
Alex-PLACET Apr 23, 2026
68eeba7
Fix benchmark
Alex-PLACET Apr 24, 2026
275182f
Refactor SearchSorted tests: consolidate assertions into reusable Che…
Alex-PLACET Apr 24, 2026
80f068d
Refactor SearchSorted tests: rename and enhance scalar search functio…
Alex-PLACET Apr 24, 2026
b24ac8d
Update SearchSorted tests: modify input values and needles for consis…
Alex-PLACET Apr 24, 2026
8919736
Add tests for sliced ChunkedRunEndEncodedValues and handle all-null c…
Alex-PLACET Apr 24, 2026
43bf7a1
Add FindNonNullValuesRange method annotation and new test for Chunked…
Alex-PLACET Apr 24, 2026
4da361c
Remove redundant check for RUN_END_ENCODED type in ReadChunkValue method
Alex-PLACET Apr 24, 2026
70f11e9
Add chunked array support for search sorted operations and enhance te…
Alex-PLACET Apr 27, 2026
a1fbab1
Formatting
Alex-PLACET Apr 27, 2026
d4e2f22
Add support for run-end encoded needles in search sorted operations
Alex-PLACET Apr 27, 2026
f6191c3
Refactor search sorted output handling to use InsertionIndexBuilder f…
Alex-PLACET May 4, 2026
18d83dc
Enhance search sorted functionality by adding tests for sliced run-en…
Alex-PLACET May 4, 2026
b36d4a4
Fix
Alex-PLACET May 7, 2026
7c1f14d
Refactor search sorted to dispatch on physical types
Alex-PLACET Jun 5, 2026
1b1b9f9
fix
Alex-PLACET Jun 5, 2026
271df6d
Reject interspersed nulls in search_sorted values
Alex-PLACET Jun 10, 2026
02a1b4a
fix casting
Alex-PLACET Jun 15, 2026
2fafd32
formatting
Alex-PLACET Jun 15, 2026
56e11a1
Add documentation and address comments
Alex-PLACET Jun 29, 2026
ad12610
wip
Alex-PLACET Jun 29, 2026
4c3abeb
fix conversion
Alex-PLACET Jun 30, 2026
e9623ad
formatting
Alex-PLACET Jul 1, 2026
451bd51
Use ChunkedArray::ComputeLogicalNullCount
Alex-PLACET Jul 3, 2026
06bcf79
fix search_sorted: handle chunked needles at top-level
Alex-PLACET Jul 3, 2026
a242791
wip 0
Alex-PLACET Jul 3, 2026
b3f75cd
wip 1
Alex-PLACET Jul 3, 2026
5b2e56c
reduce code duplication
Alex-PLACET Jul 7, 2026
2174bf8
formatting
Alex-PLACET Jul 9, 2026
1ff7761
remove deprecated test
Alex-PLACET Jul 9, 2026
cc06b2c
handle chunked REE with trailing nulls across empty chunks
Alex-PLACET Jul 23, 2026
0c1156b
revert testing submodule
Alex-PLACET Jul 23, 2026
cf5c47c
Fix ToPhysicalData for run-end encoded arrays and update tests
Alex-PLACET Jul 29, 2026
b657e17
fix compilation
Alex-PLACET Jul 30, 2026
1706c7d
Format
Alex-PLACET Aug 24, 2026
ec2a691
Fix bug with NaN values in haystack when nulls are first
pitrou Aug 27, 2026
f342add
Add float16 support
pitrou Aug 27, 2026
42ad556
Rework benchmarks
pitrou Sep 16, 2026
54c09d2
Remove obsolete lengthy comment
pitrou Sep 16, 2026
7936eaf
Architectural rework for better chunked perf and more compact impleme…
pitrou Sep 17, 2026
fd71e33
Adjust random generation of REE arrays
pitrou Sep 17, 2026
cc28c47
Add docstring
pitrou Sep 17, 2026
ff834b0
Fix lint
pitrou Sep 17, 2026
a084de3
Fix compilation error on unity builds
pitrou Sep 17, 2026
696b6be
Another compilation error
pitrou Sep 17, 2026
51bbc57
Handle needles or values with zero chunks
pitrou Sep 21, 2026
5cc09dc
Fix Valgrind failure due to GTest's default parameter printing
pitrou Sep 21, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cpp/src/arrow/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -871,6 +871,7 @@ if(ARROW_COMPUTE)
compute/kernels/vector_rank.cc
compute/kernels/vector_replace.cc
compute/kernels/vector_run_end_encode.cc
compute/kernels/vector_search_sorted.cc
compute/kernels/vector_select_k.cc
compute/kernels/vector_sort.cc
compute/kernels/vector_statistics.cc
Expand Down
2 changes: 1 addition & 1 deletion cpp/src/arrow/array/array_binary.cc
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ LargeStringArray::LargeStringArray(int64_t length,
Status LargeStringArray::ValidateUTF8() const { return internal::ValidateUTF8(*data_); }

BinaryViewArray::BinaryViewArray(std::shared_ptr<ArrayData> data) {
ARROW_CHECK_EQ(data->type->id(), Type::BINARY_VIEW);
ARROW_CHECK(is_binary_view_like(data->type->id()));
SetData(std::move(data));
}

Expand Down
30 changes: 29 additions & 1 deletion cpp/src/arrow/compute/api_vector.cc
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ using compute::FilterOptions;
using compute::NullPlacement;
using compute::RankOptions;
using compute::RankQuantileOptions;
using compute::SearchSortedOptions;

template <>
struct EnumTraits<FilterOptions::NullSelectionBehavior>
Expand Down Expand Up @@ -82,6 +83,22 @@ struct EnumTraits<DictionaryEncodeOptions::NullEncodingBehavior>
return "<INVALID>";
}
};

template <>
struct EnumTraits<SearchSortedOptions::Side>
: BasicEnumTraits<SearchSortedOptions::Side, SearchSortedOptions::Left,
SearchSortedOptions::Right> {
static std::string name() { return "SearchSortedOptions::Side"; }
static std::string value_name(SearchSortedOptions::Side value) {
switch (value) {
case SearchSortedOptions::Left:
return "Left";
case SearchSortedOptions::Right:
return "Right";
}
return "<INVALID>";
}
};
template <>
struct EnumTraits<RankOptions::Tiebreaker>
: BasicEnumTraits<RankOptions::Tiebreaker, RankOptions::Min, RankOptions::Max,
Expand Down Expand Up @@ -125,6 +142,8 @@ static auto kRunEndEncodeOptionsType = GetFunctionOptionsType<RunEndEncodeOption
static auto kArraySortOptionsType = GetFunctionOptionsType<ArraySortOptions>(
DataMember("order", &ArraySortOptions::order),
DataMember("null_placement", &ArraySortOptions::null_placement));
static auto kSearchSortedOptionsType = GetFunctionOptionsType<SearchSortedOptions>(
DataMember("side", &SearchSortedOptions::side));
static auto kSortOptionsType = GetFunctionOptionsType<SortOptions>(
CoercedDataMember("sort_keys", &SortOptions::sort_keys, &SortOptions::GetSortKeys));
static auto kPartitionNthOptionsType = GetFunctionOptionsType<PartitionNthOptions>(
Expand Down Expand Up @@ -182,12 +201,15 @@ ArraySortOptions::ArraySortOptions(SortOrder order, NullPlacement null_placement
null_placement(null_placement) {}
constexpr char ArraySortOptions::kTypeName[];

SearchSortedOptions::SearchSortedOptions(SearchSortedOptions::Side side)
: FunctionOptions(internal::kSearchSortedOptionsType), side(side) {}
constexpr char SearchSortedOptions::kTypeName[];

ARROW_SUPPRESS_DEPRECATION_WARNING
SortOptions::SortOptions(std::vector<SortKey> sort_keys)
: FunctionOptions(internal::kSortOptionsType),
sort_keys(std::move(sort_keys)),
null_placement(std::nullopt) {}

SortOptions::SortOptions(std::vector<SortKey> sort_keys,
std::optional<NullPlacement> null_placement)
: FunctionOptions(internal::kSortOptionsType),
Expand Down Expand Up @@ -277,6 +299,7 @@ void RegisterVectorOptions(FunctionRegistry* registry) {
DCHECK_OK(registry->AddFunctionOptionsType(kDictionaryEncodeOptionsType));
DCHECK_OK(registry->AddFunctionOptionsType(kRunEndEncodeOptionsType));
DCHECK_OK(registry->AddFunctionOptionsType(kArraySortOptionsType));
DCHECK_OK(registry->AddFunctionOptionsType(kSearchSortedOptionsType));
DCHECK_OK(registry->AddFunctionOptionsType(kSortOptionsType));
DCHECK_OK(registry->AddFunctionOptionsType(kPartitionNthOptionsType));
DCHECK_OK(registry->AddFunctionOptionsType(kSelectKOptionsType));
Expand Down Expand Up @@ -318,6 +341,11 @@ Result<std::shared_ptr<Array>> SelectKUnstable(const Datum& datum,
return result.make_array();
}

Result<Datum> SearchSorted(const Datum& values, const Datum& needles,
const SearchSortedOptions& options, ExecContext* ctx) {
return CallFunction("search_sorted", {values, needles}, &options, ctx);
}

Result<Datum> ReplaceWithMask(const Datum& values, const Datum& mask,
const Datum& replacements, ExecContext* ctx) {
return CallFunction("replace_with_mask", {values, mask, replacements}, ctx);
Expand Down
38 changes: 38 additions & 0 deletions cpp/src/arrow/compute/api_vector.h
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,21 @@ class ARROW_EXPORT ArraySortOptions : public FunctionOptions {
NullPlacement null_placement;
};

class ARROW_EXPORT SearchSortedOptions : public FunctionOptions {
public:
enum Side {
Left,
Right,
};

explicit SearchSortedOptions(Side side = Side::Left);
static constexpr const char kTypeName[] = "SearchSortedOptions";
static SearchSortedOptions Defaults() { return SearchSortedOptions(); }

/// Whether to return the leftmost or rightmost insertion point.
Side side;
};

class ARROW_EXPORT SortOptions : public FunctionOptions {
public:
explicit SortOptions(std::vector<SortKey> sort_keys = {});
Expand Down Expand Up @@ -598,6 +613,29 @@ Result<std::shared_ptr<Array>> SelectKUnstable(const Datum& datum,
const SelectKOptions& options,
ExecContext* ctx = NULLPTR);

/// \brief Find insertion indices that preserve sorted order.
///
/// The `values` datum must be a plain array, chunked array, or run-end encoded
/// array (including chunked run-end encoded) sorted in ascending order.
/// `needles` may be a scalar, plain array, chunked array, or run-end encoded
/// array (including chunked run-end encoded) whose logical value type matches
/// `values`.
///
/// Nulls in `values` are supported when clustered entirely at the start or the
/// end of the sorted array. Non-null needles are matched only against the
/// non-null portion of `values`. Null needles yield null outputs.
///
/// \param[in] values sorted array to search within
/// \param[in] needles scalar or array-like values to search for
/// \param[in] options selects left or right insertion semantics
/// \param[in] ctx the function execution context, optional
/// \return insertion indices as uint64 scalar or array
ARROW_EXPORT
Result<Datum> SearchSorted(
const Datum& values, const Datum& needles,
const SearchSortedOptions& options = SearchSortedOptions::Defaults(),
ExecContext* ctx = NULLPTR);

/// \brief Return the indices that would sort an array.
///
/// Perform an indirect sort of array. The output array will contain
Expand Down
1 change: 1 addition & 0 deletions cpp/src/arrow/compute/initialize.cc
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ Status RegisterComputeKernels() {
internal::RegisterVectorNested(registry);
internal::RegisterVectorRank(registry);
internal::RegisterVectorReplace(registry);
internal::RegisterVectorSearchSorted(registry);
internal::RegisterVectorSelectK(registry);
internal::RegisterVectorSort(registry);
internal::RegisterVectorRunEndEncode(registry);
Expand Down
8 changes: 8 additions & 0 deletions cpp/src/arrow/compute/kernels/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,13 @@ add_arrow_compute_test(vector_sort_test
arrow_compute_kernels_testing
arrow_compute_testing)

add_arrow_compute_test(vector_search_sorted_test
SOURCES
vector_search_sorted_test.cc
EXTRA_LINK_LIBS
arrow_compute_kernels_testing
arrow_compute_testing)

add_arrow_compute_test(vector_selection_test
SOURCES
vector_selection_test.cc
Expand All @@ -141,6 +148,7 @@ add_arrow_compute_benchmark(vector_sort_benchmark)
add_arrow_compute_benchmark(vector_partition_benchmark)
add_arrow_compute_benchmark(vector_topk_benchmark)
add_arrow_compute_benchmark(vector_replace_benchmark)
add_arrow_compute_benchmark(vector_search_sorted_benchmark)
add_arrow_compute_benchmark(vector_selection_benchmark)

# ----------------------------------------------------------------------
Expand Down
1 change: 1 addition & 0 deletions cpp/src/arrow/compute/kernels/meson.build
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ vector_kernel_benchmarks = [
'vector_partition_benchmark',
'vector_topk_benchmark',
'vector_replace_benchmark',
'vector_search_sorted_benchmark',
'vector_selection_benchmark',
]

Expand Down
Loading
Loading