镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

feat: preserve ordering through to_timestamp_seconds/millis/micros/nanos and to_unixtime - #26111

Open
haohuaijin wants to merge 2 commits into
apache:mainfrom
haohuaijin:feat/26108-timestamp-conversion-ordering
Open

haohuaijin wants to merge 2 commits into
apache:mainfrom
haohuaijin:feat/26108-timestamp-conversion-ordering

Conversation

@haohuaijin

@haohuaijin haohuaijin commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

Queries over ordered data can perform unnecessary sorts after timestamp conversions. Propagating ordering through these conversions lets the optimizer remove redundant sorts, including the to_timestamp_micros → date_bin → to_unixtime chain.

What changes are included in this PR?

  • Add ordering propagation for single-argument to_timestamp_seconds, to_timestamp_millis, to_timestamp_micros, and to_timestamp_nanos calls with integer input.
  • Add ordering propagation for single-argument to_unixtime calls with integer, Date32, Date64, or timestamp input.
  • Preserve input sort direction and NULL placement. Overflow continues to report an error.
  • Keep strict-ordering metadata conservative so precision reduction does not incorrectly eliminate sorting by subsequent keys.

What is the testing strategy for this PR?

integer_timestamp_ordering checks all four timestamp functions across integer types, sort directions, NULL placement, and unsupported inputs. ordering_matches_runtime_conversion checks to_unixtime ordering and temporal runtime conversions across date types, timestamp units, and timezones.

Tests in the existing datetime/timestamps.slt verify plans without redundant SortExec, conversion results and boundaries, the full conversion chain, and a case where precision reduction creates ties and a subsequent sorting key still requires SortExec.

Validation passed:

  • Both added unit tests.
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- datetime/timestamps.slt
  • cargo fmt --all -- --check and git diff --check
  • cargo clippy --all-targets --all-features -- -D warnings
  • uv run ./dev/rust_lint.sh

Are there any user-facing changes?

Yes. Queries using these conversions on ordered input can avoid redundant sorts. Function results and public APIs are unchanged.

@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels Oct 7, 2026
@haohuaijin haohuaijin changed the title feat: preserve ordering through to_timestamp_micros and to_unixtime feat: preserve ordering through to_timestamp_micros and to_unixtime Oct 7, 2026
@codecov-commenter

codecov-commenter commented Oct 7, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.25000% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.73%. Comparing base (a70dd34) to head (1c88c3d).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/functions/src/datetime/to_unixtime.rs 89.01% 0 Missing and 10 partials ⚠️
datafusion/functions/src/datetime/to_timestamp.rs 94.20% 0 Missing and 4 partials ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##             main   #26111    +/-   ##
========================================
  Coverage   82.72%   82.73%            
========================================
  Files        1147     1147            
  Lines      449087   449373   +286     
  Branches   449087   449373   +286     
========================================
+ Hits       371522   371770   +248     
- Misses      54917    54936    +19     
- Partials    22648    22667    +19     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@haohuaijin haohuaijin changed the title feat: preserve ordering through to_timestamp_micros and to_unixtime feat: preserve ordering through timestamp conversion functions Oct 7, 2026
@haohuaijin haohuaijin changed the title feat: preserve ordering through timestamp conversion functions feat: preserve ordering through to_timestamp_seconds/millis/micros/nanos and to_unixtime Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

preserve ordering through to_timestamp_seconds/millis/micros/nanos and to_unixtime

2 participants