Avoid target schema propagation into INSERT SELECT - #52
Open
osipovartem wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
N/A. This fixes a planner regression found by the Rustice Snowflake compatibility suite.
Rationale for this change
INSERT INTO target SELECT expression FROM VALUES ...currently propagates the target table schema into the nestedVALUESrelation. The raw values are coerced before the SELECT expression runs. For example, insertingmake_array(column1)into a list column turns the inner scalar value into a list first and producesList<List<T>>; Rustice hits the same bug whenPARSE_JSON(column1)targets a native Variant field.What changes are included in this PR?
Only provide
PlannerContext.table_schemawhen the INSERT source itself is a directVALUESbody.INSERT ... SELECTnow infers its source plan independently and continues to use the existing final projection for target-column casts and defaults.Are these changes tested?
Yes. A planner regression covers
INSERT INTO list_table SELECT make_array(column1) FROM VALUES ....cargo +1.94.0 test -p datafusion-sql --test sql_integration: 579 passedcargo +stable clippy -p datafusion-sql --all-targets --all-features -- -D warnings: passedcargo +1.94.0 fmt --all -- --check: passedThe repository-wide all-features clippy command is currently blocked outside this change by existing
datafusion-executiontests importing unstablestd::assert_matches.Are there any user-facing changes?
INSERT ... SELECTexpressions over nestedVALUESare planned against their actual source types instead of the target table types. DirectINSERT ... VALUES, including placeholder type inference, is unchanged.