Repository navigation
Bound the compressed length of oversized LZ4 blocks - #162
Merged
Merged
Conversation
With acceptOversizedBlocks enabled, LZ4BlockInputStream allocated the compressed buffer for the compressedLen from the block header with no upper bound other than Integer.MAX_VALUE. An LZ4 block cannot be larger than the LZ4 compress bound of its original length, so reject LZ4 blocks whose compressed length exceeds LZ4Utils.maxCompressedLength(originalLen) in all modes. This limits the allocation in the opt-in mode to slightly more than the original length, which is itself limited by the block size in the header. RAW blocks already require compressedLen == originalLen. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a hardening change, not a bug fix.
With
withAcceptOversizedBlocks(true),LZ4BlockInputStreamaccepts LZ4-method blocks whose compressed length is at least the original length. It growscompressedBufferto the header'scompressedLenbefore reading the payload, and the only upper bound isInteger.MAX_VALUE. GHSA-4v53-57pg-c464 and the javadoc ofwithAcceptOversizedBlocksalready document this trade-off.The opt-in mode doesn't need an unbounded value. A legitimate LZ4 block can't be larger than the LZ4 compress bound of its original length (
LZ4Utils.maxCompressedLength(originalLen), i.e.LZ4_compressBound).originalLenis already capped by the block size in the header (at most 32 MiB). This PR rejects LZ4 blocks whosecompressedLenexceeds that bound, in all modes, with the sameStream is corruptederror. Even with oversized blocks enabled, the compressed-buffer allocation is now at most slightly more than the declared block size.Details:
compressedLen < originalLenis already enforced.compressedLen == originalLen, so they need no extra check.originalLen <= compressedLen <= bound) are still accepted. The existingtestAcceptsInvalidLz4CompressedLengthInOversizedBlocksModestill passes.Builder.withAcceptOversizedBlocks.This came up while fuzzing with micronaut-fuzzing.
Test: I added
LZ4BlockStreamingTest.testRejectsLz4CompressedLengthAboveCompressBoundInOversizedBlocksMode. It checks that a header withcompressedLenofmaxCompressedLength(16) + 1orInteger.MAX_VALUEis rejected in oversized mode before the payload is read. Without the change it fails: the stream tries to read the payload and reportsStream ended prematurely. With the change,LZ4BlockStreamingTestpasses.🤖 Generated with Claude Code