Repository navigation
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces automatic stream resumption for BigQuery's Arrow stream reader (createQueryArrowStream) by retrying transient gRPC errors (such as UNAVAILABLE, DEADLINE_EXCEEDED, and INTERNAL with RST_STREAM) from the last successfully read offset up to a maximum of three times. Comprehensive unit tests are also added to verify this behavior. The review feedback highlights a critical issue regarding the use of removeAllListeners() on the gRPC stream during cancellation and retries, which can inadvertently strip internal library listeners and cause resource leaks. It is recommended to avoid removeAllListeners() and instead explicitly detach only the registered listeners (data, error, end) using named functions.
e8fd2fb to
72ed966
Compare
d4bf688 to
da4e16d
Compare
a1856e7 to
663534b
Compare
da77b2d to
c8290d3
Compare
7614972 to
9197508
Compare
8fe5111 to
35977c3
Compare
35977c3 to
9df7133
Compare
9df7133 to
35977c3
Compare
|
Rebased on the #9598–#9601 changes. Same behavior and retry rules. The retry is now a short async loop in |
35977c3 to
ec5ff11
Compare
Resume Storage Read API ReadRows streams at offset + rowsReceived when interrupted by transient gRPC errors (DEADLINE_EXCEEDED, UNAVAILABLE, or INTERNAL with RST_STREAM), up to MAX_STREAM_RETRIES consecutive attempts, waiting 100 ms before the first reopen and doubling the wait for each consecutive retry. Surface non-retryable errors or exhausted retries on the stream even after earlier batches have been emitted. Abandoned gRPC streams (on reconnect or early cancel) keep a no-op 'error' listener, because google-gax emits CANCELLED asynchronously after cancel() and an unhandled 'error' event would crash the process.
DEADLINE_EXCEEDED on ReadRows means the caller's deadline ran out, so pass it to the caller instead of reopening the stream, matching the design doc and the Python reader. Route the retry backoff through retryBackoff.wait so unit tests replace it instead of sleeping on real timers, and check the 100/200/400 ms sequence. Mark readRowsOnce async.
Reopen ReadRows at the next unread row offset without an added client-side timer on mid-stream UNAVAILABLE and RST_STREAM drops, matching ReadStream in @google-cloud/bigquery-storage and Python's ReadRowsStream. Initial connection failures when opening or reopening a ReadRows stream are already retried with exponential backoff by google-gax (retry-request).
6742228 to
ef7ca58
Compare
Long
ReadRowsstreams can drop on transient gRPC errors, which failed the whole read even though the Storage Read API can resume from any row. This reopens the stream at the next unread row.UNAVAILABLE, andINTERNALerrors caused byRST_STREAM, up to 3 times in a row. Any received batch resets the count.DEADLINE_EXCEEDEDisn't retried, since it means the caller's deadline ran out.errorlistener, since google-gax emits CANCELLED aftercancel().