Skip to content

npm: bound a tarball's header blocks, and report npm's unpackedSize - #912

Merged
schmitch merged 1 commit into
mainfrom
claude/600-npm-tarball-sizes
Nov 17, 2026
Merged

schmitch merged 1 commit into
mainfrom
claude/600-npm-tarball-sizes

Conversation

@schmitch

Copy link
Copy Markdown
Member

Fixes #600

1. Header blocks count toward npm.max_unpacked_bytes

Only the sizes the headers state counted, so a tarball of very many empty entries (512 bytes of header each, inflating about 1000:1) was bounded by the upload size alone.

Of the issue's two options I took counting the header blocks, not a cap on the number of entries: it needs no new setting and no number picked out of the air, and it bounds the thing that costs, the bytes inflated.

The blocks are counted apart from the data, against the same bound: a tarball is refused when its entries' data is over max_unpacked, as before, or when its header blocks are. Adding both into one sum would newly refuse a package whose files add up to within 512 × entries of the limit; this way no tarball accepted for its data is refused unless its headers alone exceed the bound (over 2 million entries at the default 1 GiB). The archive now inflates to at most three times the bound (data, headers, and padding, which is less than a header per entry) instead of to whatever the upload holds.

2. dist.unpackedSize is the size npm reports

It summed every entry: PAX and GNU long-name data and entries of skipped types included.

What npm sums, as checked (npm 10.9.9, node-tar 7.5.22, pacote 19.0.2):

  • Source: getContents in npm's lib/utils/tar.js adds entry.size for every entry tar.t's onentry gets. node-tar's read-entry.js marks long names and PAX headers (L, K, N, x, X, g) meta and unknown types ignore; parse.js emits neither. header.js reads a directory's size as 0.
  • Behaviour: npm pack <tarball> --json --dry-run on a crafted tarball holding package.json (30), a file (100), a directory, a symlink, a hard link, a FIFO, a file under a GNU long name (7), a file under a PAX path (7), a contiguous file (5) and an entry of unknown type Z (600 bytes) reports unpackedSize: 149. Forge said 1082.

This differs slightly from the issue's wording ("regular files only"): npm counts every entry node-tar emits, so a link or device whose header gives it a body is counted too. The same tarball plus a symlink stating 300 bytes of body gives unpackedSize: 449 in npm. No well-formed tarball has such an entry, so for those the two readings agree; I matched npm's measured value, and the regression test pins both numbers.

What does not count in unpackedSize still counts toward the bound. file_count and the manifest choice are untouched. Versions already stored keep the unpackedSize they were published with.

Tests

Both written first and seen to fail on main's reader:

  • header_blocks_count_toward_the_unpacked_bound: empty entries (files, and L, x, K, directory headers) passing a 64 KiB bound are refused; a package whose files fill the bound and one with exactly as many entries as the bound holds blocks are still read; one byte more of data is refused as before.
  • the_unpacked_size_is_the_one_npm_reports: the tarball above, 149 and 449.

Ran: cargo test -p forge-format-npm (64 + 15 pass), cargo clippy --workspace --all-targets -- -D warnings, cargo fmt --all --check, cargo xtask changelog --check.

Notes

  • No admin API change, so no web UI screen.
  • The zero block that ends the archive already counted with the data before this change (a package whose files fill the bound to the last 512 bytes is refused); left as it was.
  • While checking npm: npm pack --json's entryCount counts directories, links and FIFOs (9 for the tarball above) where Forge's dist.fileCount counts files (5). Not part of this issue; raised separately.

Generated by Claude Code

The 512-byte header blocks did not count toward npm.max_unpacked_bytes, so
a tarball of very many empty entries was bounded by the upload size alone.
They now have the same bound again, counted apart from the entries' data:
a package whose files fit the bound is read as before.

dist.unpackedSize summed every entry, the data of PAX headers and GNU long
names and entries of skipped types included. It is now what npm reports:
the sizes of the entries node-tar hands npm (`npm pack <tarball> --json`,
npm 10.9.9 with node-tar 7.5.22).

Fixes #600

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e6DJaP3vPrfBdtWXoxmth
@schmitch
schmitch added this pull request to the merge queue Nov 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 17, 2026
@schmitch
schmitch added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 724e492 Nov 17, 2026
19 checks passed
@schmitch
schmitch deleted the claude/600-npm-tarball-sizes branch November 17, 2026 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

npm tarball sizes: header blocks do not count toward the unpacked limit, and dist.unpackedSize counts more than npm's files

2 participants