fix: support ClickHouse ORDER BY WITH FILL and INTERPOLATE (#2467) - #2469
Merged
manticore-projects merged 1 commit intoAug 15, 2026
Conversation
…r#2467) Parse the ClickHouse WITH FILL modifier on ORDER BY elements together with the query-level INTERPOLATE clause, matching the upstream grammar: per element [WITH FILL] [FROM e] [TO e] [STEP e] [STALENESS e] (the step may be an INTERVAL literal), and after the ORDER BY list an optional INTERPOLATE [(col [AS expr], ...)] where the bare form fills all allowed columns. OrderByElement carries a new WithFill holder and Select (shared by PlainSelect and SetOperationList, with union hoisting) carries the interpolate column list. Rendering goes through both toString and the deparser, and the validators walk the new expressions. The keywords FILL, STEP, STALENESS and INTERPOLATE are declared non-reserved, so they remain usable as plain column names. Signed-off-by: 付典 <fudianchn@gmail.com>
Contributor
|
Thank you much! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Support ClickHouse's
ORDER BY ... WITH FILLmodifier and the query-levelINTERPOLATEclause, following the ClickHouse grammar: per element[WITH FILL] [FROM expr] [TO expr] [STEP expr] [STALENESS expr](the step may be anINTERVALliteral), and after the ORDER BY list an optionalINTERPOLATE [(col [AS expr], ...)]where the bare form fills all allowed columns.Why
SELECT d, v FROM t ORDER BY d WITH FILL FROM toDate('2024-01-01') TO toDate('2024-02-01') STEP INTERVAL 1 DAY INTERPOLATE (v AS v + 1)currently fails with a ParseException onWITH. WITH FILL is the standard ClickHouse construct for gap filling in time series, so statements using it cannot be parsed at all.How
OrderByElementcarries a newWithFillholder (its presence marks the element as filled; from/to/step/staleness are optional expressions). This mirrors ClickHouse, where the bounds belong to the order by element.Select(shared byPlainSelect,ParenthesedSelectandSetOperationList) carries theList<InterpolateElement>as a sibling oforderByElements, mirroring ClickHouse, where INTERPOLATE is a query-level attribute. The existing union hoisting fororderByElementsis extended soSELECT ... UNION SELECT ... ORDER BY x WITH FILL INTERPOLATE (...)lands on theSetOperationList.toString()and the deparser (bounds rendered through the expression visitor), andOrderByValidator/SelectValidatorwalk the new expressions.FILL,STEP,STALENESSandINTERPOLATEare declared as non-reserved keywords (inside theNonReservedWordrange), so they remain usable as plain column names (SELECT fill, step FROM t ORDER BY interpolatestill parses).Root cause
The grammar had no production for the WITH FILL modifier family and no INTERPOLATE clause; the element-level bounds and the query-level column list are modeled after the ClickHouse reference parser (bounds only parsed after
WITH FILL, in fixed FROM/TO/STEP/STALENESS order).Testing
ClickHouseTest: 9 new tests covering the issue statement, bareWITH FILL, FROM/TO/STEP,STEP INTERVALplusSTALENESS, DESC/NULLS LAST combinations, INTERPOLATE variants (multiple items, item without AS, bareINTERPOLATE), a UNION with hoisting, a subquery, and column names using the new keywords../gradlew test: 4793 tests, 0 failures (26 skipped, pre-existing).Performance
gradle jmh,
JSQLParserBenchmark.parseSQLStatementsonperformance.sql, version=latest, 10 forks × 10 iterations (100 samples) on a 32-core host:The difference is within the confidence intervals: no regression.
Notes
compileJavaccemits one additional choice-conflict warning (12 → 13) caused by the INTERPOLATE optional inPlainSelect's clause tail; the token has exactly one valid parse at that position, and the repo already carries such warnings (tracked in compileJavacc emits JavaCC warnings for shadowed literals and unreachable DATA_TYPE branches #2403).INTERPOLATE ()with an empty list is accepted and normalized to the bareINTERPOLATEform (both fill all allowed columns in ClickHouse).Fixes #2467