Skip to content

apt uploads: read a .deb's control member within a few MiB - #831

Merged
schmitch merged 1 commit into
mainfrom
claude/serene-mccarthy-2mo5sz
Nov 3, 2026
Merged

schmitch merged 1 commit into
mainfrom
claude/serene-mccarthy-2mo5sz

Conversation

@schmitch

@schmitch schmitch commented Nov 3, 2026

Copy link
Copy Markdown
Member

Fixes #815

Problem

Reading the control member of an uploaded or imported .deb (crates/formats/apt/src/deb.rs) bounded the time it took but not the memory. tar reads a GNU long name, long link or PAX header whole before it returns the entry the header describes, up to the 256 MiB the member could inflate to. The xz decoder's window grew alongside. A 39 KB package held over 500 MiB before it was refused, so a few uploads at once could get the pod OOM-killed.

Changes

  • Raw tar walk. control_in_tar reads the entries with raw(true), like the npm tarball reader. A long name, long link or PAX header is read only when its header says 64 KiB or less (MAX_EXTENSION_BYTES, the npm reader's cap); a larger one is refused from the header alone. They are applied as tar applied them: at most one of each kind per entry, and the path comes from the long name, then the PAX path, then the header. A PAX size other than the header's and GNU sparse entries are refused, because a raw walk would lay those archives out differently. Control members never contain either.
  • Inflate bound 256 MiB → 8 MiB (MAX_CONTROL_TAR_BYTES), counted up to the end of the control file. dpkg-deb sorts the member, so only conffiles and config come before control (checked with dpkg-deb 1.22.6). A builder that puts md5sums first stays under the bound up to about 70,000 files. When the bound is reached, the error names it, and a control file the bound cuts short is refused instead of being read in part.
  • Decoder limits.
    • xz: XzReader::new_mem_limit with xz -9's 64 MiB dictionary. dpkg-deb compresses the control member like the data, so -Zxz -z9 declares 64 MiB on it. lzma-rust2 grows its window only as data is decoded, so what it holds follows the 8 MiB bound.
    • zstd: an 8 MiB window, which is what level 19 declares. ruzstd returns nothing from a frame until it has decoded more than a full window, so it holds the whole declared window. With level 22's 128 MiB, a crafted package held 197 MiB here. So the zstd window limit is the memory bound.

Limitation this introduces

dpkg-deb streams control members larger than about 128 KiB. At -Zzstd -z20 to -z22 those declare windows of 32 to 128 MiB, so such packages are now refused with the control member's zstd window is larger than 8 MiB. Levels up to 19 (Ubuntu's zstd packages included) and xz at any level are read. This is documented in docs/apt.md (upload section and Limitations) and in docs/changes/2.2.0/815.limitation.md. apt is new in 2.2.0, so the bug itself never shipped and gets no fixed fragment.

Tests

  • Unit tests in deb.rs:

    • every compression, zstd now included;
    • long name, long link and PAX header refused from a header claiming 256 MiB, and read up to the cap;
    • long names and PAX paths naming the next entry, plus the refused layouts;
    • the inflate bound at its edge;
    • xz dictionaries of 8 and 64 MiB read, 96 MiB and 4 GiB refused;
    • zstd windows of 8 MiB read, 16, 32 and 128 MiB refused.
  • crates/formats/apt/tests/control_memory.rs: the issue's acceptance test. A counting global allocator, in a test binary of its own, measures what control_file holds on the issue's crafted packages (a 256 MiB GNU long name), built cheaply from concatenated xz streams, a hand-built deflate stream and zstd RLE blocks:

    Package Held before Held now
    xz 393 MiB 147 KiB
    gzip (≈ 256 MiB, per the issue) 78 KiB
    zstd, 8 MiB window – 12.5 MiB

    The test fails against the old reader.

  • Real packages built with dpkg-deb 1.22.6 and an 830 KB md5sums: gzip -z9, xz -z6/-z9, zstd -z3/-z19 are read; zstd -z22 is refused.

  • cargo test -p forge-format-apt, cargo clippy -p forge-format-apt --all-targets -D warnings, cargo fmt --check, cargo xtask changelog --check and cargo xtask docs --check pass.

No admin API changes, so no web UI screen applies.


Generated by Claude Code

Reading the control member of an uploaded or imported .deb bounded the
time it took but not the memory: `tar` read a GNU long name, long link or
PAX header whole before the entry it describes, up to the 256 MiB the
member could inflate to, and the xz decoder's window grew alongside. A
39 KB package held over 500 MiB before it was refused, so a few uploads at
once could get the instance OOM-killed.

- Walk the control tar's entries raw and read a long name, long link or
  PAX header only when it is at most 64 KiB (the npm reader's cap); apply
  them as `tar` did, refuse a PAX size other than the header's and GNU
  sparse entries.
- Let the control member inflate to 8 MiB (was 256 MiB) up to the end of
  its control file, which also bounds the xz window; name the bound when
  it is reached, and refuse a control file it cuts short.
- Give the xz decoder a memory limit (a 64 MiB dictionary, `xz -9`'s) and
  the zstd decoder an 8 MiB window (level 19's): ruzstd hands nothing out
  until it has decoded a window more, so it holds the whole window.
  dpkg-deb -Zzstd -z20 and higher are therefore refused for control
  members over about 128 KiB, documented as a limitation.

The crafted packages of the issue are refused holding 147 KiB (xz) and
78 KiB (gzip), measured by a counting allocator in a test of its own; the
reader before held 393 MiB on the xz one.

Fixes #815

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e6DJXwVipsHzkYYgSs779
@schmitch
schmitch added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit aa80eb9 Nov 3, 2026
19 checks passed
@schmitch
schmitch deleted the claude/serene-mccarthy-2mo5sz branch November 3, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

apt uploads: a 40 KB .deb makes the control parser hold about 0.5 GiB, so a few concurrent uploads OOM-kill the instance

2 participants