Skip to content

npm: ignore data after a tarball's end of archive and gzip padding, as npm does - #868

Merged
schmitch merged 1 commit into
mainfrom
claude/599-npm-tarball-trailing-data
Nov 3, 2026
Merged

schmitch merged 1 commit into
mainfrom
claude/599-npm-tarball-trailing-data

Conversation

@schmitch

@schmitch schmitch commented Nov 3, 2026

Copy link
Copy Markdown
Member

Fixes #599

What changes

Since #554, crates/formats/npm/src/tarball.rs refused non-zero bytes anywhere after the tar crate's end of archive and after gzip's trailing zero padding. npm reads less than that, so Forge refused tarballs npm installs. On a proxy repository that is a cached 502. The reader now ignores exactly what npm ignores:

  • After the two zero blocks. node-tar sets SAW_EOF at the second zero block in a row and reads nothing after it. Forge now ignores what follows, whether it is junk or a whole entry. It still inflates it within max_unpacked, as npm does, but leaves it out of dist.unpackedSize.
  • After gzip padding. Node's zlib starts a new member only when the byte after a member is non-zero. On a zero byte it stops and drops the rest of the input. Gunzip now ends there too and never inflates the rest. Before, it demanded zeros up to the end of the input.

What is still refused (#554's protection)

  • The block after a lone zero block. node-tar reads that block as a header. The tar crate ends at the first zero block, so this block must still be all zeros. An entry npm would read there (a later package/package.json with an install script) is still refused. Forge never reads fewer entries, or other ones, than npm installs.
  • Two cases kept strict. Only an all-zero block counts as the second zero block. A stream that ends partway through that block must also be zeros there, as before. node-tar ignores such a partial block, but no writer produces one, so this conservative rule keeps the parser differential closed.
  • Data right after a gzip member. If no zero byte comes first, the data is still read as another member, as Node's zlib does. Junk there still fails, as it fails for npm.

Checked against npm's own node-tar

I built six tarballs and extracted each with npm 10.9.9's bundled node-tar (tar.x, strip: 1). Forge's tarball::read returned the same package.json for each, and refused the one-lone-zero-block case:

tarball node-tar unpacks Forge
entry, 2 zero blocks, 4 KiB of 0xff a a
entry, 2 zero blocks, a second package.json a a
entry, 1 zero block, a second package.json b (install script) refused
member, \0, junk a a
member, 3000 zeros, a second member a a
two members back to back b b

Tests

  • New test ignores_data_after_the_end_of_the_archive covers junk and a whole entry after the two zero blocks, and the unpacked bound on that tail. It also checks that an entry, or junk, after a lone zero block is still refused.
  • reads_every_gzip_member now covers junk and another member after the padding being ignored, and junk right after a member being refused.
  • refuses_data_after_a_lone_zero_block (renamed) also covers junk in that block.
  • cargo fmt --all --check, cargo clippy -p forge-format-npm --all-targets -- -D warnings, cargo test -p forge-format-npm and cargo xtask changelog --check all pass.

Docs and changelog

  • docs/npm.md: the refusal list now says "data after a lone zero block".
  • Changelog fragment: docs/changes/2.2.0/599.fixed.md.

Generated by Claude Code

…s npm does

node-tar ends an archive at two zero blocks in a row and reads nothing
after them, and Node's zlib stops at the first zero byte after a gzip
member. Forge refused non-zero data in both places, so a tarball npm
installs was refused (a cached 502 through a proxy repository).

Accept and ignore what follows the second zero block (still inflated
within the unpacked bound) and what follows the gzip padding's first
zero byte. The block after a lone zero block, which node-tar reads as a
header, must still be zeros, so no entry npm reads is hidden from Forge.

Fixes #599

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e6DJZWVNoh4qBxyUBMv29
@schmitch
schmitch enabled auto-merge November 3, 2026 21:39
@schmitch
schmitch added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit 474590e Nov 3, 2026
19 checks passed
@schmitch
schmitch deleted the claude/599-npm-tarball-trailing-data branch November 3, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

npm: a tarball with bytes after its end-of-archive or after gzip's zero padding is refused, though npm installs it

2 participants