Skip to content

fix(bigquery-jdbc): record adoption features once per object - #15044

Open
Neenu1995 wants to merge 11 commits into
mainfrom
jdbc-telemetry-updates
Open

Neenu1995 wants to merge 11 commits into
mainfrom
jdbc-telemetry-updates

Conversation

@Neenu1995

@Neenu1995 Neenu1995 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

What & Why

  • Exclude driver-internal calls from usage telemetry: Route BigQueryDriver.connect startup logging, BigQueryConnection.isValid health checks, and BigQueryDatabaseMetaData internal queries through internal helpers (getMetaDataInternal, createStatementInternal, prepareStatementInternal) so driver housekeeping is not counted as user activity. Synchronize getMetaDataInternal so threads sharing a connection initialize and see a single DatabaseMetaData instance.
  • Deduplicate feature counts and fix batch error status: Record metadata retrieval and parameter binding once per Connection, ResultSet, or PreparedStatement rather than on every repeated call. In BigQueryPreparedStatement.executeBatch, record batch usage on success only and mark Storage Write failures (SQLException / RuntimeException) as STATUS_ERROR instead of STATUS_UNSPECIFIED.
  • Bound integration test memory: Cap maven-failsafe-plugin Java Virtual Machine (JVM) memory (-Xmx1536m -XX:MaxDirectMemorySize=1g -XX:+ExitOnOutOfMemoryError) and reduce ITStatementTest.testScript row counts from 500k/700k/900k to 30k/40k/50k (still above the 30k Arrow Read activation threshold) so direct-memory (memory allocated outside the Java heap) spikes fail the offending test with OutOfMemoryError instead of killing the build container (exit 137).

How It Was Tested

  • Unit tests (make unittest): Added tests in BigQueryConnectionTest, BigQueryDatabaseMetaDataTest, BigQueryDriverTest, BigQueryJsonResultSetTest, BigQueryPreparedStatementSettersTest, and BigQueryStatementTest covering internal-call exclusion, once-per-object feature counts, concurrent getMetaData(), and batch error reporting.
  • Integration tests: ITStatementTest#testScript requires Google Cloud credentials and is verified in presubmit.

Risks / Follow-ups

  • Presubmit integration tests run in a single forked JVM; if another test exceeds the new 1.5 GB heap or 1 GB direct-memory limit, it will terminate the fork with OutOfMemoryError and should be right-sized similarly.

@Neenu1995
Neenu1995 requested review from a team as code owners November 16, 2026 21:08

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request ensures that driver-internal database operations (such as connection validation, metadata queries, and pooled connection delegation) are not reported as user activity in usage telemetry. It introduces internal statement creation methods, tracks metadata retrieval and parameter binding to record them at most once, and improves error status reporting for Write API batch failures. Feedback from the review highlights a potential data race in BigQueryConnection.getMetaDataInternal() due to unsynchronized lazy initialization, and suggests replacing absolute local file paths in SKILL.md with relative paths.

@Neenu1995
Neenu1995 changed the base branch from main to turn-off-telemetry November 16, 2026 22:18
@Neenu1995
Neenu1995 requested review from a team as code owners November 16, 2026 22:18
@Neenu1995
Neenu1995 marked this pull request as draft November 16, 2026 22:19
@Neenu1995
Neenu1995 changed the base branch from turn-off-telemetry to main November 16, 2026 22:37
@Neenu1995
Neenu1995 changed the base branch from main to turn-off-telemetry November 16, 2026 22:38
… from tracked skills

It was swept into the main merge by accident; main tracks only the shared
bigquery-jdbc-testing and bigquery-jdbc-prepare-pr skills.
Base automatically changed from turn-off-telemetry to main November 17, 2026 00:48
@Neenu1995
Neenu1995 marked this pull request as ready for review November 17, 2026 15:24
@Neenu1995
Neenu1995 removed request for a team November 19, 2026 20:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant