fix(storage): reject uploadFileInChunks when the automatic abort succeeds - #9194
fix(storage): reject uploadFileInChunks when the automatic abort succeeds#9194Om-singhaI wants to merge 1 commit into
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Code Review
This pull request removes an early return statement after aborting a multipart upload in transfer-manager.ts, ensuring proper error propagation. It also updates the unit tests to correctly await asynchronous rejections and adds a new test case to verify the behavior when the abort operation itself fails. No review comments were provided, so there is no additional feedback.
…eeds When a chunked XML multipart upload failed and the automatic abort of the upload session succeeded, uploadFileInChunks returned undefined instead of throwing. Callers had no way to tell such an upload apart from a successful one, so a failed upload looked like a written file. The doc comment on the method promises an error carrying the uploadId and the map of uploaded parts, and every other failure path already throws a MultiPartUploadError. The catch block now falls through to the existing throw after a successful abort, so the original error is surfaced as a MultiPartUploadError with the uploadId and the parts map, exactly as it is when the abort itself fails or when autoAbortFailure is disabled. The existing unit test for this path used assert.doesNotThrow on a promise returning function without awaiting it, so it never observed the outcome of the promise. It now awaits assert.rejects and checks the error type, the message, the uploadId and the parts map. A second test covers the case where the abort itself fails. The neighbouring test for the disabled autoAbortFailure path had an assert.rejects call that was not awaited and now awaits it as well. Fixes googleapis#9193
03ec179 to
71f8eaf
Compare
fix(storage): reject uploadFileInChunks when the automatic abort succeeds
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #9193 🦕
What was wrong
TransferManager.uploadFileInChunksdrives an XML multipart upload: initiate, upload the parts, complete. When any of those steps fails after anuploadIdexists, the method aborts the upload session by default (autoAbortFailureunset or true). The catch block then did this:So the only outcome that did not throw was "the upload failed and the abort succeeded". The promise resolved with
undefined, and callers that treat a resolved promise as a written object carried on as if the file existed. The reporter hit exactly this and lost data silently. The method's doc comment promises the opposite ("otherwise a error containing the message, uploadId, and parts map"), and every other failure path (initiate fails, abort fails,autoAbortFailure: false) already throws aMultiPartUploadError.The fix
Remove the early
returnso a successful abort falls through to the existingthrow new MultiPartUploadError(message, uploadId, partsMap). The error now carries the original failure message plus theuploadIdand the parts map, which is the documented contract and matches what the abort failure path and theautoAbortFailure: falsepath already do. No other logic changes.Note on behaviour: a case that previously resolved now rejects. That is the point of the fix and it is what the documentation and the sample on cloud.google.com already describe, but I am calling it out explicitly so nobody is surprised. I framed this as a patch level fix rather than a breaking change because the resolved
undefinedwas never documented and could not be distinguished from success; if the maintainers would rather treat it as breaking, the commit message can be adjusted.Tests
test/transfer-manager.ts,describe('uploadFileInChunks'):should call abortUpload when a failure occurs after an uploadID is establishedpreviously usedassert.doesNotThrowon a promise returning arrow function without awaiting anything, so it passed regardless of whether the promise resolved or rejected and effectively pinned the bug. It nowawaitsassert.rejects, checks that the rejection is aMultiPartUploadErrorwhose message,uploadIdand parts map match what the fake helper reported, and checks thatabortUploadwas called once. The fake now establishes theuploadIdininitiateUpload, records a part in the parts map inuploadPart, and fails incompleteUpload, so the parts map assertion is meaningful (a non empty map is propagated).should reject with the abort error when abortUpload also failscovers the inner catch, which had no unit test before: the abort rejects, and the method rejects with aMultiPartUploadErrorcarrying the abort error's message, theuploadIdand the parts map.should reject with an error with empty uploadId and partsMap(theautoAbortFailure: falsetest) calledassert.rejectswithoutawait, so its assertion could never fail the test. It now awaits it. No other change to that test. I am happy to drop this one word change if you would rather keep the PR to the single path from the issue.Why the fakes fail in
completeUploadrather thanuploadPart:completeUploadis awaited directly by the method. A fake whoseuploadPartrejects synchronously rejects while the read loop is still waiting on the next chunk from disk, and the wrapper promise fromp-limitsits unhandled for one event loop turn untilPromise.allattaches a handler. With the tests now awaiting the result that shows up as twoPromiseRejectionHandledWarninglines in the mocha output. Failing incompleteUploadkeeps the tests deterministic and the output clean. That short unhandled window in the source is pre existing and separate from this issue; I mention it under "Out of scope" below.Verification
All runs were on
handwritten/storagewithnpm run compile:cjs -- --sourceMapfollowed bymocha build/cjs/test/transfer-manager.js(the same file the package'snpm testruns under c8, limited to the transfer manager suite; the unit tests use the sinonFakeXMLHelperharness and need no GCP credentials).main(48e0941): 44 passing. A standalone script that callsuploadFileInChunkswith a helper whoseuploadPartfails and whoseabortUploadsucceeds printedabortUpload called and succeededfollowed by the promise resolving toundefined.main(test file only changed): 44 passing, 1 failing. The failing test isshould call abortUpload when a failure occurs after an uploadID is establishedwithAssertionError [ERR_ASSERTION]: Missing expected rejection.The new abort failure test passes onmainas well, as expected, because that path already threw.PromiseRejectionHandledWarningoutput. The same standalone script now printsREJECTED: MultiPartUploadError part upload failed uploadId= upload-123 partsMap= Map(1) { 1 => 'etag-1' }.Coverage: the only removed source line is the early
return; the innercatch(abort failure) gains its first unit test, so line coverage ofsrc/transfer-manager.tsgoes up, not down.Lint
I ran the monorepo linter the way
presubmit.yamldoes (node ./bin/linter.mjs, which diffsmain...HEAD, runs ESLint with the root config, and type checks the package withtsc --noEmit). Type checking passes. Two pre existing findings are unrelated to this change and are identical onmain:src/transfer-manager.tsreports 32prettier/prettier"Delete," errors both before and after this change (only the line numbers shift by one). The file onmainis formatted withtrailingComma: "all"(the root gts 6 config) while the package's own.prettierrc.cjsresolves the package local gts 5 config withtrailingComma: "es5". My one line deletion neither adds nor removes any of these.test/transfer-manager.tscannot be parsed by that linter at all because the packagetsconfig.jsondoes not includetest/. Linting it withtsconfig.cjs.jsonas the project instead gives 0 prettier findings and three pre existing rule findings on lines this PR does not touch (duplicatefsimports at lines 41 and 42, and an explicitanyat line 639). Before this PR that run also flagged@typescript-eslint/no-floating-promisesat the un awaitedassert.rejectsin theautoAbortFailure: falsetest; that is the one wordawaitincluded here.npx prettier --check test/transfer-manager.tswith the package config passes.Out of scope, noted for maintainers
await Promise.all(promises)(that is, while the loop is still waiting on the next chunk from disk) produces an unhandled rejection on thep-limitwrapper promise for one event loop turn. Under Node's default--unhandled-rejections=throwthat terminates the process instead of reaching the catch block at all. Real part uploads take far longer than a disk read, so this is rare in practice, but it is a separate latent issue and I did not touch it here.src/transfer-manager.tsandtest/transfer-manager.ts. Neither touches the catch block changed here. fix(storage): optimize stream download throughput and eliminate write latency #9158 does reformat a large part of the test file, so whichever lands second will need a rebase of the test hunks; the one line source change is unaffected.