[CALCITE-7794] Reduce the work HepPlanner does per rule application attempt - #5272
Open
michaelbraun wants to merge 1 commit into
Open
michaelbraun wants to merge 1 commit into
michaelbraun wants to merge 1 commit into
Conversation
michaelbraun
commented
Sep 18, 2026
| * Matches the children of {@code rel} against those of {@code operand}, for a {@code rel} | ||
| * that {@code operand} is already known to match. | ||
| */ | ||
| private static boolean matchOperandsWithOperandMatched( |
Contributor
Author
There was a problem hiding this comment.
I don't like the name of this method, could use help here
michaelbraun
commented
Sep 18, 2026
| boolean forceConversions, int nMatches) { | ||
| while (iter.hasNext()) { | ||
| HepRelVertex vertex = iter.next(); | ||
| // Once per vertex: a match ends the loop below, so membership cannot change in it. |
Contributor
Author
There was a problem hiding this comment.
Would also appreciate help phrasing this better
…ttempt Rule application attempts greatly outnumber matches, so applyRule now tests the rule's operand before doing any other work, skipping the ConverterRule and CommonRelSubExprRule checks and the bindings and node-children allocations for an attempt that cannot match. The graph-membership check moves to the two callers, which try every rule against the same vertex and so need it only once, and applyRules returns early when there is no rule at all. Adds a benchmark that plans a range of query shapes through rules that fire, over varying numbers of rule collections and rules per collection, so planning changes can be measured against realistic queries. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
michaelbraun
force-pushed
the
CALCITE-7794
branch
from
September 18, 2026 01:43
472ce5e to
01eb104
Compare
michaelbraun
marked this pull request as ready for review
September 18, 2026 01:43
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Summary
HepPlannerapplies a rule collection by trying every rule against every vertex, so rule application attempts can outnumber matches. Three pieces of avoidable work sat on that path.applyRuletestedgraph.vertexSet().contains(vertex)on every call. Both call sites try every rule against the same vertex object in a loop that only exits early on a match, so membership cannot change within it: 50 rules meant 50 set lookups where one suffices. The test moves to the two callers, andapplyRulesreturns early for an empty rule collection so the hoisted test is not paid when there is no rule to apply.applyRuleallocated thebindingslist and thenodeChildrenmap before callingmatchOperands, which rejects on its first statement, so for the large majority of attempts both were allocated and discarded.ConverterRuleandCommonRelSubExprRulebranches ran before the operand was tested, traversing a vertex's parents for attempts that could never match.applyRulenow tests the operand first, andmatchOperandsis split into a thin entry point making that test plusmatchOperandsWithOperandMatchedassuming it, so it is not repeated for attempts that pass. The matching logic is unchanged and recursive descent still goes through the checking entry point.Measurements
HepQueryVarietyBenchmark, added here, plans queries converted from SQL and measuressetRootplusfindBestExp.collectionsis how manyaddRuleCollectioncalls the program makes,rulesPerCollhow many rules each holds,firing%the share of the rule set able to rewrite these queries. Means over four rounds and five query shapes, percent faster than unmodifiedHepPlanner:A collection of one rule gains least, which follows from which change applies: hoisting the membership test saves nothing there, since one rule means one lookup either way. Above one rule per collection that test starts to pay, which is most of what this changes. Gains fall as more of the rule set can fire, since transformation work then grows while the rejection cost does not.
The non-firing rules are real ones for
INTERSECT,MINUS,MATCH_RECOGNIZE,TABLESAMPLEand exchange, which none of these queries contain, so the attempts they add reject at the operand depths real rules reject at rather than all at the top level. Plans are identical whatever the non-firing share, confirming those rules never fire.Test plan
HepPlannerTest,RelOptRulesTest,MaterializedViewRelOptRulesTest,CombineRelOptRulesTest— 1064 tests, 0 failurescheckstyleMain,checkstyleJmh,autostyleCheckclean./gradlew build🤖 Generated with Claude Code