[ohmayr]

Extends bigquery.createQueryArrowStream(query) to wait for unfinished queries (jobComplete === false) before streaming Arrow results from the job's default read stream (projects/{project}/locations/{location}/jobs/{job}/streams/_default).

• Adds the ArrowQuery options type (beta, like the method) with timeoutMs.
• Passes timeoutMs to jobs.query and fails if jobComplete === false when a caller-provided timeoutMs expires.
• Polls job.getQueryResults with maxResults: 0 and autoPaginate: false until the job completes without fetching row data, then streams all batches via ReadRows starting at offset = 0.
• Emits a 0-row RecordBatch when reading from offset = 0 and the stream closes without any record batches.
• When a result read from offset = 0 has no rows, reads its schema from the job's destination table (jobs.get, then CreateReadSession), because streams/_default sends nothing for an empty result. This is skipped if the consumer has destroyed the stream.
• Re-runs the Arrow system tests under BIGQUERY_PICOSECOND_SUPPORT (high_precision_timestamp.ts) to cover a TIMESTAMP(12) column during the job wait.

[gemini-code-assist]

Code Review

This pull request adds support for waiting on unfinished BigQuery queries when executing createQueryArrowStream. It introduces a polling mechanism (waitForJobCompletion) using job.getQueryResults with maxResults: 0 to wait for job completion before streaming results, and adds configuration for timeoutMs. Additionally, comprehensive unit and system tests are added to cover these scenarios. The feedback highlights a critical issue in waitForJobCompletion where polling is executed in a tight loop without any delay, which could lead to API rate limiting. It is recommended to introduce a delay using setTimeout and implement proper cleanup on stream destruction.

[shivanee-p]

It's great to see the progress!

I have some concerns with the polling logic. I think having it be asynchronous would be really beneficial.

[shivanee-p]

See previous comment on #9598 regarding private API variables

[ohmayr]

Fixed. _ended is gone here too; the stream only ends in one place (see #9598).

[shivanee-p]

It makes sense regarding the polling mechanism. However there are a few concerns:

• poll() is called immediately when jobComplete is false, but we should have a slight delay to prevent throwing rate limiting errors
• I think transforming this code to be asynchronous would be a good call. Using an async/await loop would be preferred to avoid nested recursive chaining

[ohmayr]

Made it an async loop. I left out a client-side delay: getQueryResults blocks on the server for about 10 s per call while the job runs (measured: 20 polls in 201 s on a long job), so the loop already polls about 6 times a minute, and a delay would only add latency after the job finishes. This matches how query() waits today.

[ohmayr]

Following up on the polling delay: each jobs.getQueryResults call with maxResults: 0 waits on the server for up to about 10 s while the job runs, so polling again right away doesn't spin. A client-side delay would only add latency after the job finishes.

[ohmayr]

Rebased on the #9598 changes. Waiting behaves the same; only how it's written changed:

• Polling is an await loop instead of a recursive callback (no client-side delay; see the L573 reply).
• The destination-schema lookup uses await instead of nested callbacks.
• The empty-result fallback runs after await streamReadRows(...) instead of inside the gRPC 'end' handler, and the stream only ends in one place, so the _ended checks are gone.
• Added a small pushBatches helper and used it for the first-page push loop from feat(bigquery): stream multi-page Arrow results from default job stream #9598 as well, so the two loops don't duplicate each other.

[shivanee-p]

We should also check if (ctx.stream.destroyed) in the event that the caller destroys the stream

[ohmayr]

Added in b31bbd7: readDestinationSchema returns without creating a read session once the stream is destroyed, with a test.

[shivanee-p]

These stubs should be changed to be promises to mimic the behavior of the actual implementation.

[ohmayr]

Converted in b31bbd7: stubJobGetQueryResults now takes an async function. I left stubCreateJob and the bq.request stubs callback-based, because createQueryJob calls createJob with a callback internally and request is callback-based.