Repository navigation
Conversation
The ZigZag decoder trusted the page header's var-int byte count: it pushed that many bytes into a list one at a time, wrote one 7-bit group per continuation byte into a 10-byte decode buffer without a bound, and indexed the byte list after it ran out. Validate both header var-ints, reject a zigzag_length_ larger than the remaining input, stop at the ten groups a valid int64 value can carry, and treat an exhausted list as corruption. decode() now reports errors through read_int32/read_int64. Valid pages decode identically.
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Malformed varint headers can trigger undefined behavior, and oversized encoded integers are silently decoded as zero instead of rejected as corruption.
2 open issues
What changed in this PR
This PR hardens Apache TsFile’s C++ ZigZag decoder against malformed page data and propagates decoding failures through read_int32 and read_int64.
Changes:
- Checks header-read errors and declared byte lengths before loading encoded data.
- Bounds continuation-group writes and detects exhausted value data.
- Adds malformed-stream regression tests and updates existing tests to assert read status.
| File | Description |
|---|---|
cpp/test/encoding/zigzag_codec_test.cc |
Adds corruption cases and migrates decoding assertions to status-returning APIs. |
cpp/test/encoding/encoding_coverage_test.cc |
Updates reset coverage to check read status and decoded values. |
cpp/src/encoding/zigzag_decoder.h |
Adds input and buffer bounds checks and propagates decoding errors. |
Review effort: Balanced
Comment on lines
+113
to
124
| int read_header(common::ByteStream& in) { | ||
| int ret = common::E_OK; | ||
| if (RET_FAIL( | ||
| common::SerializationUtil::read_var_uint(zigzag_length_, in))) { | ||
| return common::E_TSFILE_CORRUPTED; | ||
| } | ||
| if (RET_FAIL( | ||
| common::SerializationUtil::read_var_uint(int_length_, in))) { | ||
| return common::E_TSFILE_CORRUPTED; | ||
| } | ||
| return common::E_OK; | ||
| } |
Comment on lines
+132
to
+136
| if (UNLIKELY(num_of_sorts_of_zigzag_ >= DECODE_ARR_CAPACITY)) { | ||
| // More continuation bytes than any valid value can carry; writing | ||
| // one more group would run past zigzag_decode_arr_. | ||
| return false; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

The ZigZag decoder trusted the page header's byte count: it read that many bytes into a list one at a time, wrote one 7-bit group per continuation byte into a 10-byte buffer without a bound, and indexed the byte list after it ran out. Validate both header var-ints, reject a declared length larger than the remaining input, stop at the ten groups an int64 value can carry, and treat an exhausted list as corruption.
decode()now reports errors throughread_int32/read_int64. Valid pages decode identically.Tests:
ZigzagDecoderTest.RejectsDeclaredLengthBeyondInput,RejectsValueRunningPastDeclaredLength,RejectsValueWithTooManyContinuationBytes,RejectsEmptyAndHeaderlessStreams. Full C++ suite passes.