Refactor unzip functionality to limit concurrency and improve error handling in copyStream#4455
Merged
Merged
Conversation
Contributor
|
Great start @shati-patel . I've pushed a commit to also add timeouts, in case something goes wrong with the writing. That will give us an extra layer or protection. |
josefs
force-pushed
the
shati-patel/fix-cli-extraction
branch
from
ac3da10 to
d23ed63
Compare
josefs
force-pushed
the
shati-patel/fix-cli-extraction
branch
from
d23ed63 to
c593313
Compare
Contributor
|
I've added a test for the timeout and reworked the other test a bit. This is ready for review now. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses reported stalls during CodeQL CLI installation/update by making archive extraction more resilient: extraction concurrency is capped, stream copy failures propagate correctly, and extraction will abort with a clear error if progress stops for too long (using the configured download timeout when extracting the CLI distribution).
Changes:
- Refactor
copyStreamto usestream/promises.pipeline(better error propagation + optional abort signal support). - Add an idle-timeout abort mechanism to unzip operations, and thread the timeout through sequential and concurrent unzip entry points.
- Limit unzip concurrency (cap at 4) to reduce simultaneous writes during extraction, and add unit tests covering the new error/timeout handling.
Show a summary per file
| File | Description |
|---|---|
| extensions/ql-vscode/test/unit-tests/common/unzip.test.ts | Adds unit tests validating copyStream rejection on write errors and abort-on-idle behavior. |
| extensions/ql-vscode/src/common/unzip.ts | Refactors stream copy to pipeline, adds idle-timeout abort for unzip, and plumbs timeout/signal through extraction. |
| extensions/ql-vscode/src/common/unzip-concurrently.ts | Caps concurrent extraction tasks to avoid overwhelming storage; forwards timeout to unzip core. |
| extensions/ql-vscode/src/codeql-cli/distribution.ts | Passes the configured download timeout into extraction so stalled extraction aborts predictably. |
| extensions/ql-vscode/CHANGELOG.md | Documents the user-facing fix for hangs during CLI extraction and the new timeout behavior. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 5/5 changed files
- Comments generated: 1
- Review effort level: Low
Comment on lines
+256
to
+260
| } catch (e) { | ||
| if (signal.aborted) { | ||
| throw new Error( | ||
| `Timed out while extracting archive: no progress was made for ${timeoutSeconds} seconds. ` + | ||
| "This can happen if a file cannot be written, for example because of slow or networked storage, " + |
kaspersv
reviewed
Jul 16, 2026
kaspersv
approved these changes
Jul 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4453
Discussed with @josefs and tried various other fixes locally (like limiting the number of available cores). Eventually settled on this change to
copyStream.I installed the dev VSIX build locally and can confirm that the extraction now works smoothly 🎉
Note: this still needs a changelog note and potentially some tidy-up