Skip to content

gh-158841: Fix crash in pickle.load() when read() returns a bytes subclass - #158846

Open
drakeo338 wants to merge 1 commit into
python:mainfrom
drakeo338:claude/158841-fix
Open

drakeo338 wants to merge 1 commit into
python:mainfrom
drakeo338:claude/158841-fix

Conversation

@drakeo338

@drakeo338 drakeo338 commented Oct 5, 2026 •

Copy link
Copy Markdown

pickle.load() crashes when a file object's read() returns a bytes subclass and the data is large enough to need several reads. _Unpickler_ReadFromFile() grows the result with _PyBytes_Resize(), which is only valid for exact bytes. The fix copies a subclass instance into an exact bytes object first. A regression test and a NEWS entry are included.

The new test segfaults on a debug build without the fix and passes with it. test_pickle, test_pickletools, test_copyreg and test_picklebuffer pass.

AI disclosure: written with Claude Code assistance; I reviewed the change and ran the tests.

Fixes #158841.

AI-assisted: I used Claude Code to help write this change. I reviewed it and ran test_pickle, test_pickletools, test_copyreg and test_picklebuffer on a debug build.

…es subclass

_Unpickler_ReadFromFile() resized the object returned by read() in place, which is invalid for a bytes subclass. Copy it into an exact bytes object first.
@error-3317

Copy link
Copy Markdown

Could we verify that every possible path to _PyBytes_Resize() has an exact bytes object at that point? The new conversion is guarded by cursize < n, which is probably sufficient for the current chunked-read path, but the safety invariant is that _PyBytes_Resize() must never receive a bytes subclass.

@drakeo338

Copy link
Copy Markdown
Author

Four _PyBytes_Resize() calls exist in _pickle.c, and three act on bytes the pickler or load_counted_binbytes allocates itself. The fourth, in _Unpickler_ReadFromFile, runs only inside while (cursize < n), and the new conversion uses the same condition, so a subclass becomes exact bytes before any resize. A non-bytes read() result, such as a bytearray, already raised SystemError and is unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_pickle crashes when read() returns a bytes subclass for large payloads

2 participants