gh-158567: Optimize PyFloat_Pack*() functions by vstinner · Pull Request #158483 · python/cpython

vstinner · 2026-09-30T06:56:37Z

PyFloat_Pack4() and PyFloat_Pack8() avoid temporary buffer in the native byte order.

Use _Py_bswapXX() functions to reverse bytes.

Issue: Optimize PyFloat_Pack*() functions using _Py_bswap16/32/64() functions #158567

PyFloat_Pack4() and PyFloat_Pack8() avoid temporary buffer in the native byte order. Use _Py_bswapXX() functions to reverse bytes.

vstinner · 2026-09-30T06:59:51Z

Benchmark:

import pyperf, sys, _testcapi
runner = pyperf.Runner()
le = int(sys.byteorder == 'little')
for d in (1.0, float('nan')):
    runner.bench_func(f'pack2({d}) native', _testcapi.float_pack, 2, d, le)
    runner.bench_func(f'pack4({d}) native', _testcapi.float_pack, 4, d, le)
    runner.bench_func(f'pack8({d}) native', _testcapi.float_pack, 8, d, le)
    be = int(not le)
    runner.bench_func(f'pack2({d}) byteswap', _testcapi.float_pack, 2, d, be)
    runner.bench_func(f'pack4({d}) byteswap', _testcapi.float_pack, 4, d, be)
    runner.bench_func(f'pack8({d}) byteswap', _testcapi.float_pack, 8, d, be)

Result:

Benchmark	ref	change
pack2(1.0) native	68.1 ns	56.2 ns: 1.21x faster
pack4(1.0) native	58.0 ns	49.2 ns: 1.18x faster
pack8(1.0) native	52.8 ns	49.7 ns: 1.06x faster
pack2(1.0) byteswap	67.2 ns	59.2 ns: 1.13x faster
pack4(1.0) byteswap	60.1 ns	51.4 ns: 1.17x faster
pack8(1.0) byteswap	59.1 ns	50.8 ns: 1.16x faster
pack2(nan) native	54.2 ns	51.8 ns: 1.05x faster
pack4(nan) native	54.7 ns	53.0 ns: 1.03x faster
pack8(nan) native	57.9 ns	52.4 ns: 1.10x faster
pack2(nan) byteswap	60.5 ns	53.5 ns: 1.13x faster
pack4(nan) byteswap	54.1 ns	53.1 ns: 1.02x faster
pack8(nan) byteswap	54.9 ns	53.0 ns: 1.04x faster
Geometric mean	(ref)	1.11x faster

For example, PyFloat_Pack8() x86-64 assembly code before:

   0x00000000004a6250 <+0>:	test   esi,esi
   0x00000000004a6252 <+2>:	jne    0x4a62a0 <PyFloat_Pack8+80>

   0x00000000004a6254 <+4>:	movq   rax,xmm0
   0x00000000004a6259 <+9>:	mov    rdx,rax
   0x00000000004a625c <+12>:	mov    BYTE PTR [rdi+0x7],al
   0x00000000004a625f <+15>:	shr    rdx,0x10
   0x00000000004a6263 <+19>:	mov    BYTE PTR [rdi+0x6],ah
   0x00000000004a6266 <+22>:	mov    BYTE PTR [rdi+0x5],dl
   0x00000000004a6269 <+25>:	mov    rdx,rax
   0x00000000004a626c <+28>:	shr    rdx,0x18
   0x00000000004a6270 <+32>:	mov    BYTE PTR [rdi+0x4],dl
   0x00000000004a6273 <+35>:	mov    rdx,rax
   0x00000000004a6276 <+38>:	shr    rdx,0x20
   0x00000000004a627a <+42>:	mov    BYTE PTR [rdi+0x3],dl
   0x00000000004a627d <+45>:	mov    rdx,rax
   0x00000000004a6280 <+48>:	shr    rdx,0x28
   0x00000000004a6284 <+52>:	mov    BYTE PTR [rdi+0x2],dl
   0x00000000004a6287 <+55>:	mov    rdx,rax
   0x00000000004a628a <+58>:	shr    rax,0x38
   0x00000000004a628e <+62>:	shr    rdx,0x30
   0x00000000004a6292 <+66>:	mov    BYTE PTR [rdi],al
   0x00000000004a6294 <+68>:	xor    eax,eax
   0x00000000004a6296 <+70>:	mov    BYTE PTR [rdi+0x1],dl
   0x00000000004a6299 <+73>:	ret

   0x00000000004a62a0 <+80>:	movsd  QWORD PTR [rdi],xmm0
   0x00000000004a62a4 <+84>:	xor    eax,eax
   0x00000000004a62a6 <+86>:	ret

Assembly code after:

   0x00000000004a6220 <+0>:	movq   rax,xmm0
   0x00000000004a6225 <+5>:	test   esi,esi
   0x00000000004a6227 <+7>:	mov    rdx,rax
   0x00000000004a622a <+10>:	bswap  rdx
   0x00000000004a622d <+13>:	cmove  rax,rdx
   0x00000000004a6231 <+17>:	mov    QWORD PTR [rdi],rax
   0x00000000004a6234 <+20>:	xor    eax,eax
   0x00000000004a6236 <+22>:	ret

The conditional jump is replaced with more efficient cmove, and bytes are swapped by bswap rdx instruction.

vstinner · 2026-09-30T07:00:11Z

cc @skirpichev

skirpichev

LGTM

Though, not sure if the second case in Pack2 does make sense, see comment.

vstinner · 2026-09-30T13:09:59Z

@skirpichev: I pushed a change to also use _Py_bswap16()+memcpy() at the end of PyFloat_Pack2().

Update benchmark results (CPU isolated, after running sudo python3 -m pyperf system tune):

Benchmark	ref	bswap
pack4(1.0) native	95.2 ns	94.4 ns: 1.01x faster
pack8(1.0) native	93.1 ns	91.2 ns: 1.02x faster
pack8(1.0) byteswap	94.2 ns	89.4 ns: 1.05x faster
pack2(nan) native	95.7 ns	92.6 ns: 1.03x faster
pack4(nan) native	94.9 ns	94.3 ns: 1.01x faster
pack8(nan) native	94.0 ns	89.7 ns: 1.05x faster
pack4(nan) byteswap	94.6 ns	94.0 ns: 1.01x faster
pack8(nan) byteswap	94.3 ns	90.4 ns: 1.04x faster
Geometric mean	(ref)	1.02x faster

Benchmark hidden because not significant (4): pack2(1.0) native, pack2(1.0) byteswap, pack4(1.0) byteswap, pack2(nan) byteswap

This time it's no longer "1.11x faster", but at least, it's not slower on any benchmark: it's either faster or as fast :-)

Esp. for le case performance gain is 1.05x, maybe a noise.

Oh sure. It's really hard to measure the speedup, the difference is really tiny and can be lost in noise. I'm using CPU isolation on Linux to reduce the noise.

Details

$ python3 -m pyperf show ref.json bswap.json -q
ref
===

pack2(1.0) native: Mean +- std dev: 110 ns +- 11 ns
pack4(1.0) native: Mean +- std dev: 95.2 ns +- 1.5 ns
pack8(1.0) native: Mean +- std dev: 93.1 ns +- 0.9 ns
pack2(1.0) byteswap: Mean +- std dev: 109 ns +- 7 ns
pack4(1.0) byteswap: Mean +- std dev: 94.5 ns +- 0.6 ns
pack8(1.0) byteswap: Mean +- std dev: 94.2 ns +- 1.6 ns
pack2(nan) native: Mean +- std dev: 95.7 ns +- 4.7 ns
pack4(nan) native: Mean +- std dev: 94.9 ns +- 1.1 ns
pack8(nan) native: Mean +- std dev: 94.0 ns +- 6.2 ns
pack2(nan) byteswap: Mean +- std dev: 94.8 ns +- 5.5 ns
pack4(nan) byteswap: Mean +- std dev: 94.6 ns +- 0.7 ns
pack8(nan) byteswap: Mean +- std dev: 94.3 ns +- 1.4 ns

bswap
=====

pack2(1.0) native: Mean +- std dev: 108 ns +- 1 ns
pack4(1.0) native: Mean +- std dev: 94.4 ns +- 0.7 ns
pack8(1.0) native: Mean +- std dev: 91.2 ns +- 6.4 ns
pack2(1.0) byteswap: Mean +- std dev: 108 ns +- 1 ns
pack4(1.0) byteswap: Mean +- std dev: 95.3 ns +- 5.7 ns
pack8(1.0) byteswap: Mean +- std dev: 89.4 ns +- 0.9 ns
pack2(nan) native: Mean +- std dev: 92.6 ns +- 0.8 ns
pack4(nan) native: Mean +- std dev: 94.3 ns +- 0.5 ns
pack8(nan) native: Mean +- std dev: 89.7 ns +- 1.1 ns
pack2(nan) byteswap: Mean +- std dev: 93.5 ns +- 2.9 ns
pack4(nan) byteswap: Mean +- std dev: 94.0 ns +- 0.8 ns
pack8(nan) byteswap: Mean +- std dev: 90.4 ns +- 2.5 ns

skirpichev

One nitpick.

vstinner · 2026-10-01T12:08:58Z

Windows and macOS CI failed, unrelated failures. I just re-run these 2 jobs.

Oh, unrelated failure: test_external_inspection failed on "Windows (free-threading) / Build and test (x64, tail-call)" with OSError: [WinError 299] Only part of a ReadProcessMemory or WriteProcessMemory request was completed:

FAIL: test_tlbc_cache_refresh_after_slot_fill (test.test_external_inspection.TestGetStackTrace.test_tlbc_cache_refresh_after_slot_fill)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\a\cpython\cpython\Lib\test\test_external_inspection.py", line 2446, in test_tlbc_cache_refresh_after_slot_fill
    self.assertEqual(
    ~~~~~~~~~~~~~~~~^
        result.returncode, 0,
        ^^^^^^^^^^^^^^^^^^^^^
        f"stdout: {result.stdout}\nstderr: {result.stderr}",
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: 1 != 0 : stdout: 
stderr: OSError: [WinError 299] Only part of a ReadProcessMemory or WriteProcessMemory request was completed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 40, in <module>
    cached = lines(u, 2)
  File "<string>", line 18, in lines
    traces = u.get_stack_trace()
OSError: ReadProcessMemory failed for PID 9500 at address 0x1 (size 80, partial read 0 bytes): Windows error 299

Also, the "macOS (free-threading) / build and test (macos-26)" job interrupted. I didn't see that recently, it's surprising: == Tests result: INTERRUPTED ==.

bedevere-bot · 2026-10-01T13:01:22Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Stable Clang Installed 3.x (tier-3) has failed when building commit eb30e3d.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1645/builds/3480) and take a look at the build logs.
Check if the failure is related to this commit (eb30e3d) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1645/builds/3480

Failed tests:

test_buffer

Failed subtests:

test_ndarray_index_getitem_single - test.test_buffer.TestBufferProtocol.test_ndarray_index_getitem_single
test_ndarray_format_strides - test.test_buffer.TestBufferProtocol.test_ndarray_format_strides

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x.clang-installed/build/target/lib/python3.16/test/test_buffer.py", line 1533, in test_ndarray_index_getitem_single
    self.assertEqual(nd[i], items[i])
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
AssertionError: Tuples differ: (36832.0, 40512.0) != (36768.0, 40512.0)


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x.clang-installed/build/target/lib/python3.16/test/test_buffer.py", line 1378, in test_ndarray_format_strides
    self.verify(nd, obj=None,
    ~~~~~~~~~~~^^^^^^^^^^^^^^
                itemsize=itemsize, fmt=fmt, readonly=True,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                ndim=1, shape=shape, strides=strides,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                lst=items[::step])
                ^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x.clang-installed/build/target/lib/python3.16/test/test_buffer.py", line 855, in verify
    self.assertEqual(rep, lst)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^
AssertionError: Lists differ: [(746[15 chars] (-24736.0, -42464.0), (7984.0, -39104.0), (27472.0, -26608.0)] != [(746[15 chars] (-24736.0, -42464.0), (7984.0, -39104.0), (27472.0, -26576.0)]


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x.clang-installed/build/target/lib/python3.16/test/test_buffer.py", line 1378, in test_ndarray_format_strides
    self.verify(nd, obj=None,
    ~~~~~~~~~~~^^^^^^^^^^^^^^
                itemsize=itemsize, fmt=fmt, readonly=True,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                ndim=1, shape=shape, strides=strides,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                lst=items[::step])
                ^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x.clang-installed/build/target/lib/python3.16/test/test_buffer.py", line 855, in verify
    self.assertEqual(rep, lst)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^
AssertionError: Lists differ: [(-53[55 chars] (-44544.0, -56000.0, -20464.0), (-24224.0, -10232.0, 23648.0)] != [(-53[55 chars] (-44544.0, -56000.0, -20464.0), (-24224.0, -10216.0, 23648.0)]

bedevere-bot · 2026-10-01T13:02:30Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Stable LTO + PGO 3.x (tier-3) has failed when building commit eb30e3d.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1627/builds/3447) and take a look at the build logs.
Check if the failure is related to this commit (eb30e3d) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1627/builds/3447

Summary of the results of the build (if available):

Click to see traceback logs

Note: switching to 'eb30e3d9d4b523e3f8de2cd1485c9ae9a996ab03'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at eb30e3d9d4b Optimize PyFloat_Pack*() functions (#158483)
Switched to and reset branch 'main'

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make[2]: [Makefile:3461: clean-retain-profile] Error 1 (ignored)
Parser/lexer/string.c: In function ‘finish_ftstring_expr’:
Parser/lexer/string.c:86:20: warning: variable ‘previous_end’ set but not used [-Wunused-but-set-variable=]
   86 |         _PyTok_Off previous_end = state->expr_span.start;
      |                    ^~~~~~~~~~~~
during RTL pass: split1
Objects/floatobject.c: In function ‘PyFloat_Pack2’:
Objects/floatobject.c:2014:1: internal compiler error: Segmentation fault
 2014 | }
      | ^
/usr/libexec/gcc/s390x-redhat-linux/16/cc1 -quiet -I ./Include/internal -I ./Include/internal/mimalloc -I . -I ./Include -D NDEBUG -D Py_BUILD_CORE Objects/floatobject.c -quiet -dumpdir Objects/ -dumpbase floatobject.c -dumpbase-ext .c -mbackchain -mtune=z14 -march=z13 -m64 -mzarch -g -g -O3 -Wsign-compare -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -std=c11 -fno-strict-overflow -fno-semantic-interposition -flto -fuse-linker-plugin -ffat-lto-objects -fvisibility=hidden -fprofile-generate -fprofile-update=atomic -o /tmp/ccSOQonY.s
Please submit a full bug report, with preprocessed source.
See <https://bugzilla.redhat.com/> for instructions.
The bug is not reproducible, so it is likely a hardware or OS problem.
make[2]: *** [Makefile:3367: Objects/floatobject.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [Makefile:1000: profile-gen-stamp] Error 2
make: *** [Makefile:1012: profile-run-stamp] Error 2

bedevere-bot · 2026-10-01T13:03:57Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Stable LTO 3.x (tier-3) has failed when building commit eb30e3d.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1654/builds/3396) and take a look at the build logs.
Check if the failure is related to this commit (eb30e3d) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1654/builds/3396

Summary of the results of the build (if available):

Click to see traceback logs

Note: switching to 'eb30e3d9d4b523e3f8de2cd1485c9ae9a996ab03'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at eb30e3d9d4b Optimize PyFloat_Pack*() functions (#158483)
Switched to and reset branch 'main'

Parser/lexer/string.c: In function ‘finish_ftstring_expr’:
Parser/lexer/string.c:86:20: warning: variable ‘previous_end’ set but not used [-Wunused-but-set-variable=]
   86 |         _PyTok_Off previous_end = state->expr_span.start;
      |                    ^~~~~~~~~~~~
In function ‘bytearray_resize_storage’,
    inlined from ‘bytearray_resize_lock_held’ at Objects/bytearrayobject.c:339:9,
    inlined from ‘bytearray_append_impl’ at Objects/bytearrayobject.c:2201:9,
    inlined from ‘bytearray_append’ at Objects/clinic/bytearrayobject.c.h:1248:20:
Objects/bytearrayobject.c:259:13: warning: ‘memmove’ specified bound 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]
  259 |             memmove(self->ob_bytes + moved, old_start + moved, remaining);
      |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
during RTL pass: split1
Objects/floatobject.c: In function ‘PyFloat_Pack2’:
Objects/floatobject.c:2014:1: internal compiler error: Segmentation fault
 2014 | }
      | ^
/usr/libexec/gcc/s390x-redhat-linux/16/cc1 -quiet -I ./Include/internal -I ./Include/internal/mimalloc -I . -I ./Include -D NDEBUG -D Py_BUILD_CORE Objects/floatobject.c -quiet -dumpdir Objects/ -dumpbase floatobject.c -dumpbase-ext .c -mbackchain -mtune=z14 -march=z13 -m64 -mzarch -g -g -O3 -Wsign-compare -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -std=c11 -fno-strict-overflow -flto -fuse-linker-plugin -ffat-lto-objects -fvisibility=hidden -o /tmp/ccxqU1rz.s
Please submit a full bug report, with preprocessed source.
See <https://bugzilla.redhat.com/> for instructions.
The bug is not reproducible, so it is likely a hardware or OS problem.
make: *** [Makefile:3367: Objects/floatobject.o] Error 1
make: *** Waiting for unfinished jobs....

vstinner · 2026-10-01T14:33:19Z

s390x Fedora Stable LTO 3.x: Oh! gcc -O3 crashed when building floatobject.c!

during RTL pass: split1
Objects/floatobject.c: In function ‘PyFloat_Pack2’:
Objects/floatobject.c:2014:1: internal compiler error: Segmentation fault
 2014 | }
      | ^

Please submit a full bug report, with preprocessed source.
See <https://bugzilla.redhat.com/> for instructions.

test_buffer fails on s390x Clang buildbot workers. Example:

FAIL: test_ndarray_format_strides (test.test_buffer.TestBufferProtocol.test_ndarray_format_strides)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x.clang-installed/build/target/lib/python3.16/test/test_buffer.py", line 1378, in test_ndarray_format_strides
    self.verify(nd, obj=None,
    ~~~~~~~~~~~^^^^^^^^^^^^^^
                itemsize=itemsize, fmt=fmt, readonly=True,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                ndim=1, shape=shape, strides=strides,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                lst=items[::step])
                ^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x.clang-installed/build/target/lib/python3.16/test/test_buffer.py", line 855, in verify
    self.assertEqual(rep, lst)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^
AssertionError: Lists differ: [(-53[55 chars] (-44544.0, -56000.0, -20464.0), (-24224.0, -10232.0, 23648.0)] != [(-53[55 chars] (-44544.0, -56000.0, -20464.0), (-24224.0, -10216.0, 23648.0)]
First differing element 3:
(-24224.0, -10232.0, 23648.0)
(-24224.0, -10216.0, 23648.0)
  [(-53536.0, 23600.0, 34592.0),
   (-3188.0, -60672.0, 20304.0),
   (-44544.0, -56000.0, -20464.0),
-  (-24224.0, -10232.0, 23648.0)]
?                 ^^
+  (-24224.0, -10216.0, 23648.0)]
?                 ^^

vstinner · 2026-10-01T15:18:48Z

s390x Fedora Stable LTO 3.x: Oh! gcc -O3 crashed when building floatobject.c!

I reported the GCC crash to GCC bug tracker as: https://bugzilla.redhat.com/show_bug.cgi?id=2544649.

vstinner · 2026-10-01T16:02:00Z

test_buffer fails randomly on s390x with Clang. I ran ./python -m test -v test_buffer 6 times in a row. It only failed 4 times (it passed 2 times):

Run 1:

FAIL: test_ndarray_format_strides (test.test_buffer.TestBufferProtocol.test_ndarray_format_strides)
FAIL: test_ndarray_index_setitem_single (test.test_buffer.TestBufferProtocol.test_ndarray_index_setitem_single)

Run 2:

FAIL: test_ndarray_multidim (test.test_buffer.TestBufferProtocol.test_ndarray_multidim)

Run5:

FAIL: test_ndarray_index_getitem_single (test.test_buffer.TestBufferProtocol.test_ndarray_index_getitem_single)

Run 6:

FAIL: test_ndarray_index_getitem_single (test.test_buffer.TestBufferProtocol.test_ndarray_index_getitem_single)

vstinner · 2026-10-01T16:32:29Z

I generated test cases to trigger the bug on s390x with clang:

import _testcapi

BIG_ENDIAN = 0
LITTLE_ENDIAN = 1

pack = _testcapi.float_pack

def format_bytes(b):
    return ''.join(f'\\x{byte:02x}' for byte in b)

def test(d, expected, endian):
    result = pack(2, d, endian)
    if result != expected:
        result = format_bytes(result)
        expected = format_bytes(expected)
        print(f"test failed: {d=} result='{result}' expected='{expected}' {endian=}")

test(float.fromhex('0x1.5f07c01032e34p-1'), b'\x39\x7c', BIG_ENDIAN)
test(float.fromhex('0x1.5f07c01032e34p-1'), b'\x7c\x39', LITTLE_ENDIAN)

test(float.fromhex('0x1.3ee08ca31ea80p-4'), b'\x2c\xfc', BIG_ENDIAN)
test(float.fromhex('0x1.3ee08ca31ea80p-4'), b'\xfc\x2c', LITTLE_ENDIAN)

test(float.fromhex('0x1.5ee5f0edf590cp-3'), b'\x31\x7c', BIG_ENDIAN)
test(float.fromhex('0x1.5ee5f0edf590cp-3'), b'\x7c\x31', LITTLE_ENDIAN)

Output whe Python is built with clang -O3:

test failed: d=0.6856060046934034 result='\x7e\x39' expected='\x7c\x39' endian=1
test failed: d=0.07785086571122157 result='\xfe\x2c' expected='\xfc\x2c' endian=1
test failed: d=0.1713370154017827 result='\x7e\x31' expected='\x7c\x31' endian=1

If I apply #158568 fix, the clang bug goes away.

Optimize PyFloat_Pack*() functions

aa273f8

PyFloat_Pack4() and PyFloat_Pack8() avoid temporary buffer in the native byte order. Use _Py_bswapXX() functions to reverse bytes.

vstinner added the skip news label Sep 30, 2026

bedevere-app Bot added the awaiting core review label Sep 30, 2026

vstinner added the skip issue label Sep 30, 2026

skirpichev approved these changes Sep 30, 2026

View reviewed changes

Comment thread Objects/floatobject.c Outdated

Also use _Py_bswap16() at the end of PyFloat_Pack2()

89bb000

skirpichev reviewed Oct 1, 2026

View reviewed changes

Comment thread Objects/floatobject.c Outdated

Remove redundant comments

2c87001

vstinner enabled auto-merge (squash) October 1, 2026 11:20

vstinner merged commit eb30e3d into python:main Oct 1, 2026
90 of 94 checks passed

vstinner deleted the float_pack branch October 1, 2026 12:25

bedevere-app Bot removed the awaiting core review label Oct 1, 2026

vstinner removed the skip issue label Oct 1, 2026

vstinner changed the title ~~Optimize PyFloat_Pack*() functions~~ gh-158567: Optimize PyFloat_Pack*() functions Oct 1, 2026

bedevere-app Bot mentioned this pull request Oct 1, 2026

Optimize PyFloat_Pack*() functions using _Py_bswap16/32/64() functions #158567

Open

Uh oh!

Conversation

vstinner commented Sep 30, 2026 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Sep 30, 2026

Uh oh!

vstinner commented Sep 30, 2026

Uh oh!

skirpichev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vstinner commented Sep 30, 2026

Uh oh!

skirpichev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vstinner commented Oct 1, 2026

Uh oh!

Uh oh!

bedevere-bot commented Oct 1, 2026

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Oct 1, 2026

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Oct 1, 2026

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

vstinner commented Oct 1, 2026

Uh oh!

vstinner commented Oct 1, 2026

Uh oh!

vstinner commented Oct 1, 2026

Uh oh!

vstinner commented Oct 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vstinner commented Sep 30, 2026 •

edited by bedevere-app Bot

Loading