Skip to content

fix(bigquery-jdbc): gate Storage Write API on direct VALUES inserts and align columns - #15086

Open
Neenu1995 wants to merge 1 commit into
fix/swa-2-unified-write-conversionfrom
fix/swa-3-direct-insert-gating
Open

Neenu1995 wants to merge 1 commit into
fix/swa-2-unified-write-conversionfrom
fix/swa-3-direct-insert-gating

Conversation

@Neenu1995

Copy link
Copy Markdown
Contributor
  • Fix silent expression/UDF bypass (b/587228731) and AppendSerializationError when EnableWriteAPI=true is used with SQL expressions, CASTs, or UDFs inside INSERT ... VALUES (...).
  • Add a zero-regex whitelist parser (parseDirectInsertValues) so only single-row INSERT [INTO] <table> [(<cols>)] VALUES (?, ...) [;] statements with bare ? placeholders use the Storage Write API; all other INSERT forms fall back automatically to BigQuery SQL DML.
  • Align reordered column lists (resolveInsertSchema), reject partial/duplicate/pseudo-column lists, honor DefaultDataset project matching (resolveTargetProject), retry unbound dry runs once when tables are created after prepareStatement(), and fall back to SQL inside active transactions or _SESSION temp tables.

Testing

  • make lint, mvn -B -ntp verify -DskipTests -DskipITs, and make unittest (250/250 targeted Write API / parameter / statement tests passing).

@Neenu1995
Neenu1995 requested review from a team as code owners November 19, 2026 20:50
@Neenu1995
Neenu1995 added this pull request to stack #15087 November 19, 2026 20:51

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances the BigQuery JDBC driver's support for the Storage Write API (SWA) by introducing a parser to recognize single-row direct INSERT ... VALUES statements, resolving target projects and schemas, and validating SWA eligibility based on transaction and session states. It also fixes a bug in BigQueryStatement regarding default dataset configuration and adds comprehensive unit tests. The reviewer suggests moving the assignment of 'insertMetadataEvaluated' to the beginning of the try block in executeBatch() to prevent redundant dry-run API requests if an exception is thrown.

Comment on lines +826 to +832
try {
captureInsertMetadata(
getQueryStatistics(getWriteBatchJobConfiguration(this.batchParameters.peek())));
QueryStatistics statistics = describePositionalParameterQuery(this.currentQuery);
this.insertMetadataEvaluated = true;
if (statistics != null) {
applyInferredParameterTypes(statistics);
captureInsertMetadata(statistics);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Setting this.insertMetadataEvaluated = true; at the beginning of the try block ensures that even if describePositionalParameterQuery throws an exception (e.g., due to a persistent syntax or permission error), we do not repeatedly attempt the dry run on subsequent executeBatch() calls. This avoids redundant API requests to BigQuery.

        try {
          this.insertMetadataEvaluated = true;
          QueryStatistics statistics = describePositionalParameterQuery(this.currentQuery);
          if (statistics != null) {
            applyInferredParameterTypes(statistics);
            captureInsertMetadata(statistics);
          }
        }

…nd align columns

Stack: PR 3 of 3 (base: `fix/swa-2-unified-write-conversion`, stacked on #15084 -> #15085)

### Summary
- Fix silent expression/UDF bypass (`b/587228731`) and `AppendSerializationError` when `EnableWriteAPI=true` is used with SQL expressions, `CAST`s, or UDFs inside `INSERT ... VALUES (...)`.
- Add a zero-regex whitelist parser (`parseDirectInsertValues`) so only single-row `INSERT [INTO] <table> [(<cols>)] VALUES (?, ...) [;]` statements with bare `?` placeholders use the Storage Write API; all other `INSERT` forms fall back automatically to BigQuery SQL DML.
- Align reordered column lists (`resolveInsertSchema`), reject partial/duplicate/pseudo-column lists, honor `DefaultDataset` propagation and project matching (`BigQueryStatement.generateBigQuerySettings`, `resolveTargetProject`), skip positional dry runs in Legacy SQL mode (`getUseLegacySql`), retry unbound dry runs at most once when tables are created after `prepareStatement()`, and fall back to SQL inside active transactions or `_SESSION` temp tables.

### Testing
- `make lint`, `mvn -B -ntp verify -DskipTests -DskipITs`, `mvn -q -Denforcer.skip=true -Dclirr.skip=true test-compile`, and `make unittest` (284 targeted tests passing; full suite 1305/1306 non-skipped passing, 1 pre-existing `BigQueryJdbcOAuthUtilityTest` P12 env failure; ITs require GCP credentials).
@Neenu1995
Neenu1995 force-pushed the fix/swa-3-direct-insert-gating branch from 4d16c0b to 22875e8 Compare November 20, 2026 00:45

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant