Skip to content

gh-158316: Avoid second buffer fetch on bytearray_decode - #158485

Merged
vstinner merged 4 commits into
python:mainfrom
johng:gh-158316-bytearray-decode
Oct 1, 2026
Merged

vstinner merged 4 commits into
python:mainfrom
johng:gh-158316-bytearray-decode

Conversation

@johng

@johng johng commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

@bedevere-app bedevere-app Bot added the type-feature A feature request or enhancement label Sep 30, 2026

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please unregister the error handler when done:

diff --git a/Lib/test/test_bytes.py b/Lib/test/test_bytes.py
index b923f31178c..97460f1fc57 100644
--- a/Lib/test/test_bytes.py
+++ b/Lib/test/test_bytes.py
@@ -18,6 +18,7 @@
 import textwrap
 import threading
 import unittest
+from _codecs import _unregister_error as _codecs_unregister_error
 
 import test.support
 from test import support
@@ -2203,6 +2204,7 @@ def handler(exc):
             self.assertRaises(BufferError, b.clear)
             self.assertRaises(BufferError, b.append, 0)
             return ('?', exc.end)
+        self.addCleanup(_codecs_unregister_error, 'test.bytearray_decode_resize')
         codecs.register_error('test.bytearray_decode_resize', handler)
         for encoding in 'utf-8', 'utf-8-sig':
             with self.subTest(encoding=encoding):

Comment thread Objects/bytearrayobject.c
exports to prevent the storage from changing during the decode. */
self->ob_exports++;
PyObject *res = PyUnicode_Decode(PyByteArray_AS_STRING(self),
Py_SIZE(self), encoding, errors);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I wanted to suggest replacing Py_SIZE() with PyByteArray_GET_SIZE() but I see that it uses an atomic operation on Free Threading, which is not needed in a critical section. Hum. PyByteArray_GET_SIZE() usage in this file is not really consistent anyway.

Comment thread Lib/test/test_bytes.py Outdated
self.assertEqual(
b.decode(encoding, 'test.bytearray_decode_resize'),
'ab?cd')
self.assertEqual(b, b'ab\xffcd')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's be confident in the implementation: checking the bytearray for modification only once at the end should be enough :-)

Suggested change
self.assertEqual(b, b'ab\xffcd')
self.assertEqual(b, b'ab\xffcd')

@johng
johng requested a review from vstinner September 30, 2026 18:08
Comment thread Lib/test/test_bytes.py
Comment thread Misc/NEWS.d/next/Core_and_Builtins/2026-09-30-10-12-41.gh-issue-158316.Yv3kQe.rst Outdated
Co-authored-by: Victor Stinner <vstinner@python.org>

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vstinner
vstinner enabled auto-merge (squash) October 1, 2026 19:51
@vstinner
vstinner merged commit a4f28a5 into python:main Oct 1, 2026
54 checks passed
@vstinner

vstinner commented Oct 1, 2026

Copy link
Copy Markdown
Member

Merged, thanks for the optimization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type-feature A feature request or enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants