Skip to content

Avoid lifting CREATE_FILTERs across binding-changing operators - #20

Merged
JP-Reddy merged 1 commit into
robust-sql:mainfrom
jianhongshi123:dev/fix_create_filter_lifting
Aug 8, 2026
Merged

Avoid lifting CREATE_FILTERs across binding-changing operators#20
JP-Reddy merged 1 commit into
robust-sql:mainfrom
jianhongshi123:dev/fix_create_filter_lifting

Conversation

@jianhongshi123

Copy link
Copy Markdown
Contributor

CREATE_FILTER lifting could go across operators that change column bindings. This caused the LOGICAL_FILTER projection map to no longer match the column bindings of the deeper LOGICAL_GET, resulting in binding errors.

This PR stops CREATE_FILTER lifting when the column bindings change.

@JP-Reddy
JP-Reddy merged commit 34b0ef6 into robust-sql:main Aug 8, 2026
12 checks passed
@JP-Reddy

JP-Reddy commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

lgtm! Can you document the operators that break these bindings?

@JP-Reddy

JP-Reddy commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

And also list the queries that were impacted from this issue.

@JP-Reddy

JP-Reddy commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

We might want to revisit this later. Lifting above the FILTER operators have meaningful gains in the JOB queries. We would want to figure out why the bindings are inconsistent and try to address them

@jianhongshi123

Copy link
Copy Markdown
Contributor Author

lgtm! Can you document the operators that break these bindings?

The old rewrite is guaranteed to be safe only if every operator between FILTER and the deepest CREATE_FILTER is unary, preserves column bindings, and can safely commute with FILTER. Binding-changing operators such as PROJECTION and AGGREGATE can invalidate the column bindings, while multi-child operators such as DELIM_JOIN and MATERIALIZED_CTE break the linear-chain assumption of the old rewrite.

For example, in Q18, an AGGREGATE lies between the FILTER and the deepest CREATE_FILTER. The FILTER is a HAVING predicate that depends on an aggregate result. The old rewrite effectively changes FILTER not on grouping key(AGGREGATE(X)) into AGGREGATE(FILTER not on grouping key(X)), which can cause both binding error and query result change.

And also list the queries that were impacted from this issue.

The affected TPC-H queries are Q02, Q07, Q17, Q18, and Q20.

We might want to revisit this later. Lifting above the FILTER operators have meaningful gains in the JOB queries. We would want to figure out why the bindings are inconsistent and try to address them

A follow-up fix is implemented in #26. It remaps the input bindings of the moved operators and without moving the FILTER across any non-CREATE/PROBE operators.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants