Is your feature request related to a problem or challenge?
Queries over ordered data can require an unnecessary sort after applying timestamp conversion functions, even when the conversions preserve the input ordering.
For example, given a table ordered by micros ASC NULLS LAST:
SELECT to_unixtime(
date_bin(
INTERVAL '1 second',
to_timestamp_micros(micros),
TIMESTAMP '1970-01-01'
)
) AS bucket
FROM ordered_timestamps
ORDER BY bucket ASC NULLS LAST;
The conversion chain preserves ordering, but the resulting plan currently includes a SortExec.
Describe the solution you'd like
Allow DataFusion to propagate ordering through:
to_timestamp_seconds, to_timestamp_millis, to_timestamp_micros, and to_timestamp_nanos with a single integer argument.
to_unixtime with a single integer, Date32, Date64, or timestamp argument.
Preserve the input sort direction and NULL placement so redundant sorts can be removed. Integer conversions report overflow rather than introducing NULLs.
Precision reduction can produce equal output values, so sorting by additional keys must remain when their ordering cannot be established.
Describe alternatives you've considered
Keep the existing explicit sort. This produces correct results but performs unnecessary work when the converted values are already ordered.
Additional context
Implementation: #26111.
Is your feature request related to a problem or challenge?
Queries over ordered data can require an unnecessary sort after applying timestamp conversion functions, even when the conversions preserve the input ordering.
For example, given a table ordered by
micros ASC NULLS LAST:The conversion chain preserves ordering, but the resulting plan currently includes a
SortExec.Describe the solution you'd like
Allow DataFusion to propagate ordering through:
to_timestamp_seconds,to_timestamp_millis,to_timestamp_micros, andto_timestamp_nanoswith a single integer argument.to_unixtimewith a single integer, Date32, Date64, or timestamp argument.Preserve the input sort direction and NULL placement so redundant sorts can be removed. Integer conversions report overflow rather than introducing NULLs.
Precision reduction can produce equal output values, so sorting by additional keys must remain when their ordering cannot be established.
Describe alternatives you've considered
Keep the existing explicit sort. This produces correct results but performs unnecessary work when the converted values are already ordered.
Additional context
Implementation: #26111.