Repository navigation
feat: preserve ordering through to_timestamp_seconds/millis/micros/nanos and to_unixtime - #26111
Open
haohuaijin wants to merge 2 commits into
Open
haohuaijin wants to merge 2 commits into
haohuaijin wants to merge 2 commits into
Conversation
to_timestamp_micros and to_unixtime
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #26111 +/- ##
========================================
Coverage 82.72% 82.73%
========================================
Files 1147 1147
Lines 449087 449373 +286
Branches 449087 449373 +286
========================================
+ Hits 371522 371770 +248
- Misses 54917 54936 +19
- Partials 22648 22667 +19 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
to_timestamp_micros and to_unixtimeto_timestamp_seconds/millis/micros/nanos and to_unixtime
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
to_timestamp_seconds/millis/micros/nanosandto_unixtime#26108Rationale for this change
Queries over ordered data can perform unnecessary sorts after timestamp conversions. Propagating ordering through these conversions lets the optimizer remove redundant sorts, including the
to_timestamp_micros→date_bin→to_unixtimechain.What changes are included in this PR?
to_timestamp_seconds,to_timestamp_millis,to_timestamp_micros, andto_timestamp_nanoscalls with integer input.to_unixtimecalls with integer, Date32, Date64, or timestamp input.What is the testing strategy for this PR?
integer_timestamp_orderingchecks all four timestamp functions across integer types, sort directions, NULL placement, and unsupported inputs.ordering_matches_runtime_conversionchecksto_unixtimeordering and temporal runtime conversions across date types, timestamp units, and timezones.Tests in the existing
datetime/timestamps.sltverify plans without redundantSortExec, conversion results and boundaries, the full conversion chain, and a case where precision reduction creates ties and a subsequent sorting key still requiresSortExec.Validation passed:
cargo test -p datafusion-sqllogictest --test sqllogictests -- datetime/timestamps.sltcargo fmt --all -- --checkandgit diff --checkcargo clippy --all-targets --all-features -- -D warningsuv run ./dev/rust_lint.shAre there any user-facing changes?
Yes. Queries using these conversions on ordered input can avoid redundant sorts. Function results and public APIs are unchanged.