You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Deduplicate Legacy/Ansi/Try integer SUM accumulators in sum_int.rs #6976
native/spark-expr/src/agg_funcs/sum_int.rs has six accumulator implementations, Legacy/Ansi/Try for both the row (SumIntegerAccumulator*) and grouped (SumIntGroupsAccumulator*) forms, and they share almost all of their code:
Each update_batch defines a local generic sum loop and then repeats the same four-arm Int64/Int32/Int16/Int8 dispatch with the same unsupported-type error (row accumulators around L119-154, L190-227, L273-316; groups accumulators at L404-473, L550-622, L711-793).
evaluate, state, convert_to_state and size are identical between the Legacy and Ansi groups accumulators, and merge_batch differs only in the add.
The real difference between the modes is one operation: add_wrapping (Legacy), add_checked mapped to an overflow error (Ansi), and add_checked mapped to None (Try). Try also tracks has_all_nulls, which is a legitimate difference.
Adding an integer type, or fixing filter or null handling, means editing six places, and the Try variant is the one most likely to drift.
Describe the potential solution
Hoist the type dispatch into one helper (e.g. dispatch_int_array(values, |arr| ...)) so the Int8 to Int64 match exists once.
Write one generic sum loop that takes the add operation (impl Fn(i64, i64) -> Option<i64>, or an OverflowMode trait), with Legacy, Ansi and Try passing their own add.
Make the groups accumulator generic over the mode so Legacy and Ansi share evaluate/state/size/merge_batch, and keep Try's has_all_nulls handling as a separate impl or mode.
Activity