Skip to content

fix: support RANGE window frames over TIME keys - #24515

Open
fornwall wants to merge 2 commits into
apache:mainfrom
fornwall:range-window-over-time
Open

fix: support RANGE window frames over TIME keys#24515
fornwall wants to merge 2 commits into
apache:mainfrom
fornwall:range-window-over-time

Conversation

@fornwall

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

Window functions ordered by Time32 or Time64 currently fail with an internal error, even when no frame clause is specified. ORDER BY defaults to RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, whose free bounds only require comparisons, and time values are orderable.

Finite RANGE offsets over time remain unsupported because time and interval arithmetic wraps around the 24-hour clock, so DataFusion's frame bound computation (current_value ± offset) cannot produce meaningful bounds. PostgreSQL supports such offsets through dedicated non-wrapping in_range logic, which is left as possible future work.

What changes are included in this PR?

  • Treat Time32 and Time64 as comparison-only RANGE frame key types.
  • Continue rejecting finite PRECEDING and FOLLOWING offsets during planning.
  • Document time keys alongside other orderable, comparison-only types.

Are these changes tested?

Yes. SQL logic tests cover all four Arrow time units, duplicate peer values, dictionary-wrapped time keys, aggregate and ranking windows, ascending and descending order, explicit free bounds, and finite offsets on both bound sides.

The full datafusion-sqllogictest suite and repository lint checks pass.

Are there any user-facing changes?

Yes. Free RANGE frames over TIME order keys now return results instead of an internal error. Finite offsets are still rejected, now with a clear planning error instead of the internal error. There are no public API or breaking changes.


AI usage: Created with Claude Code and Opus 5. I have reviewed the code and made modifications where it made sense.

Signed-off-by: Fredrik Fornwall <fredrik@fornwall.net>
@github-actions github-actions Bot added documentation Improvements or additions to documentation optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt) labels Aug 20, 2026
@fornwall fornwall changed the title fix: support range windows over time keys fix: support RANGE window frames over TIME keys Aug 20, 2026
@github-actions github-actions Bot added the development-process Related to development process of DataFusion label Aug 20, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.29%. Comparing base (5091b42) to head (8d19799).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24515      +/-   ##
==========================================
+ Coverage   81.27%   81.29%   +0.01%     
==========================================
  Files        1116     1116              
  Lines      395073   395460     +387     
  Branches   395073   395460     +387     
==========================================
+ Hits       321102   321474     +372     
+ Misses      55176    55174       -2     
- Partials    18795    18812      +17     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment on lines +7012 to +7020
# Finite offsets over time are rejected for both the start and end bound.
query error DataFusion error: type_coercion\ncaused by\nError during planning: RANGE with offset PRECEDING/FOLLOWING is not supported for ORDER BY type Time64\(µs\)
SELECT COUNT(*) OVER (ORDER BY x RANGE BETWEEN INTERVAL '1' HOUR PRECEDING AND CURRENT ROW)
FROM (VALUES (arrow_cast('01:00:00', 'Time64(Microsecond)'))) t(x)

query error DataFusion error: type_coercion\ncaused by\nError during planning: RANGE with offset PRECEDING/FOLLOWING is not supported for ORDER BY type Time64\(µs\)
SELECT COUNT(*) OVER (ORDER BY x RANGE BETWEEN CURRENT ROW AND INTERVAL '1' HOUR FOLLOWING)
FROM (VALUES (arrow_cast('01:00:00', 'Time64(Microsecond)'))) t(x)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @fornwall. Any reason why this should not be supported for Time? They both work with Postgres and DuckDB:

SELECT COUNT(*) OVER (
  ORDER BY x RANGE BETWEEN INTERVAL '1' HOUR PRECEDING AND CURRENT ROW)
FROM (
  VALUES
    (TIME '01:00:00'),
    (TIME '02:00:00'),
    (TIME '04:00:00'),
    (TIME '05:00:00')
) t(x);
 count
-------
     1
     2
     1
     2

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nuno-faria We should support that as well!

But it requires some work to handle the non-wrapping around midnight part (ordinary time +/- interval wraps), so I think that's better in a follow up PR. Does that sound ok?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-process Related to development process of DataFusion documentation Improvements or additions to documentation optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Window functions fail when ORDER BY uses a TIME column

3 participants