Skip to content

fix(cpp): reject negative and oversized string lengths when reading - #1127

Open
ColinLeeo wants to merge 1 commit into
apache:developfrom
ColinLeeo:read-var-str-length
Open

ColinLeeo wants to merge 1 commit into
apache:developfrom
ColinLeeo:read-var-str-length

Conversation

@ColinLeeo

Copy link
Copy Markdown
Contributor

String readers used the length from the stream for mem_alloc and buffer indexing without validation. Reject negative lengths with E_TSFILE_CORRUPTED and lengths beyond the remaining bytes with E_PARTIAL_READ (the code the truncated read returned before, so chunk/page header parsing still retries with a larger file window) before allocating. The same guards apply to read_str(std::string&), read_str(String&, pa) and read_mystring(). Valid strings read as before.

Tests: ByteStreamTest.VarStringRejectsNegativeLength, VarStringRejectsLengthBeyondStream, FixedStringRejectsNegativeLength. Full C++ suite passes.

read_var_str() took the length from the stream and used it for
mem_alloc(len + 1) and tmp_buf[len] without validation. A negative
length wraps to a huge allocation request or indexes before the buffer,
and a large length is allocated in full before the short read is
detected.

Reject negative lengths with E_TSFILE_CORRUPTED and lengths beyond the
remaining bytes with E_PARTIAL_READ before allocating. E_PARTIAL_READ is
what the truncated read returned before, so chunk/page header parsing
keeps retrying with a larger file window. The same guards apply to
read_str(std::string&), read_str(String&, pa) and read_mystring().
Valid strings read as before.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the string-deserialization hardening changes.

No open issues 🎉

What changed in this PR

This PR hardens C++ string deserialization by validating decoded lengths before allocation or buffer indexing, while preserving retry behavior for incomplete input.

Changes:

  • Reject negative string lengths with E_TSFILE_CORRUPTED.
  • Return E_PARTIAL_READ when a declared length exceeds the remaining bytes.
  • Add regression tests for negative and oversized variable-length and fixed-width string lengths.
File Description
cpp/​test/​common/​allocator/​byte_stream_test.cc Adds malformed-length regression tests and checks that negative lengths leave output unchanged.
cpp/​src/​common/​allocator/​byte_stream.h Adds pre-allocation length validation to standard-string and arena-backed string readers.

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants