Skip to content

fix(cpp): bound ZigZag page length and continuation bytes - #1128

Open
ColinLeeo wants to merge 1 commit into
apache:developfrom
ColinLeeo:zigzag-bounds
Open

ColinLeeo wants to merge 1 commit into
apache:developfrom
ColinLeeo:zigzag-bounds

Conversation

@ColinLeeo

Copy link
Copy Markdown
Contributor

The ZigZag decoder trusted the page header's byte count: it read that many bytes into a list one at a time, wrote one 7-bit group per continuation byte into a 10-byte buffer without a bound, and indexed the byte list after it ran out. Validate both header var-ints, reject a declared length larger than the remaining input, stop at the ten groups an int64 value can carry, and treat an exhausted list as corruption. decode() now reports errors through read_int32/read_int64. Valid pages decode identically.

Tests: ZigzagDecoderTest.RejectsDeclaredLengthBeyondInput, RejectsValueRunningPastDeclaredLength, RejectsValueWithTooManyContinuationBytes, RejectsEmptyAndHeaderlessStreams. Full C++ suite passes.

The ZigZag decoder trusted the page header's var-int byte count: it
pushed that many bytes into a list one at a time, wrote one 7-bit group
per continuation byte into a 10-byte decode buffer without a bound, and
indexed the byte list after it ran out.

Validate both header var-ints, reject a zigzag_length_ larger than the
remaining input, stop at the ten groups a valid int64 value can carry,
and treat an exhausted list as corruption. decode() now reports errors
through read_int32/read_int64. Valid pages decode identically.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Malformed varint headers can trigger undefined behavior, and oversized encoded integers are silently decoded as zero instead of rejected as corruption.

2 open issues
What changed in this PR

This PR hardens Apache TsFile’s C++ ZigZag decoder against malformed page data and propagates decoding failures through read_int32 and read_int64.

Changes:

  • Checks header-read errors and declared byte lengths before loading encoded data.
  • Bounds continuation-group writes and detects exhausted value data.
  • Adds malformed-stream regression tests and updates existing tests to assert read status.
File Description
cpp/​test/​encoding/​zigzag_codec_test.cc Adds corruption cases and migrates decoding assertions to status-returning APIs.
cpp/​test/​encoding/​encoding_coverage_test.cc Updates reset coverage to check read status and decoded values.
cpp/​src/​encoding/​zigzag_decoder.h Adds input and buffer bounds checks and propagates decoding errors.

Review effort: Balanced

Comment on lines +113 to 124
int read_header(common::ByteStream& in) {
int ret = common::E_OK;
if (RET_FAIL(
common::SerializationUtil::read_var_uint(zigzag_length_, in))) {
return common::E_TSFILE_CORRUPTED;
}
if (RET_FAIL(
common::SerializationUtil::read_var_uint(int_length_, in))) {
return common::E_TSFILE_CORRUPTED;
}
return common::E_OK;
}
Comment on lines +132 to +136
if (UNLIKELY(num_of_sorts_of_zigzag_ >= DECODE_ARR_CAPACITY)) {
// More continuation bytes than any valid value can carry; writing
// one more group would run past zigzag_decode_arr_.
return false;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants