Avoid Clang bug in UBSan CI

[StanFromIreland]

Since 4c79929 upgraded Clang, we've been hit by llvm/llvm-project#179695. For example, in https://github.com/python/cpython/actions/runs/35756221220/job/106842645848 which took 10 minutes to compile.

Other jobs that compile with Clang aren't affected, as they are all tail-calling, and the LLVM fix was backported to 21 for Darwin.

CC @lucascolley (since this may affect your builds as well?)

[vstinner]

Ah, 2 tests failed on Sanitizers / TSan (free-threading):

2 tests failed:
    test_asyncio.test_taskgroups
    test_concurrent_futures.test_process_pool

(...)

test_no_stale_references (test.test_concurrent_futures.test_process_pool.ProcessPoolSpawnProcessPoolExecutorTest.test_no_stale_references) ... Timeout (0:15:00)!
Thread 0x00007f5109263f40 [python] (most recent call first):
  File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 416 in _recv
  File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 451 in _recv_bytes
  File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 260 in recv
  File "/home/runner/work/cpython/cpython/Lib/multiprocessing/managers.py", line 570 in start
  File "/home/runner/work/cpython/cpython/Lib/multiprocessing/context.py", line 58 in Manager
  File "/home/runner/work/cpython/cpython/Lib/test/test_concurrent_futures/util.py", line 55 in setUp

(...)

ERROR: test_taskgroup_cancel_children (test.test_asyncio.test_taskgroups.TestTaskGroup.test_taskgroup_cancel_children)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython/Lib/test/test_asyncio/test_taskgroups.py", line 1141, in test_taskgroup_cancel_children
    async with asyncio.TaskGroup() as tg:
               ~~~~~~~~~~~~~~~~~^^
  File "/home/runner/work/cpython/cpython/Lib/asyncio/taskgroups.py", line 75, in __aexit__
    return await self._aexit(et, exc)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/cpython/cpython/Lib/asyncio/taskgroups.py", line 177, in _aexit
    raise propagate_cancellation_error
  File "/home/runner/work/cpython/cpython/Lib/asyncio/taskgroups.py", line 125, in _aexit
    await self._on_completed_fut
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython/Lib/asyncio/runners.py", line 128, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/runner/work/cpython/cpython/Lib/asyncio/base_events.py", line 725, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/home/runner/work/cpython/cpython/Lib/test/test_asyncio/test_taskgroups.py", line 1140, in test_taskgroup_cancel_children
    async with asyncio.timeout(1):
               ~~~~~~~~~~~~~~~^^^
  File "/home/runner/work/cpython/cpython/Lib/asyncio/timeouts.py", line 115, in __aexit__
    raise TimeoutError from exc_val
TimeoutError

[vstinner]

Do you have timings before/after this change on Clang jobs?

[StanFromIreland]

Do you have timings before/after this change on Clang jobs?

Before this patch, looking at CI runs building takes around 8-10 minutes, with the patch it takes around 1 minute.

However, this also slows down running the tests so much that we hit the 60 minute timeout. I can't reproduce that locally, so I'm not sure what's going on.

[vstinner]

Can you try -O2? According to llvm/llvm-project#179695 (comment), -O2 is not affected by the bug. It may make the tests a little big faster.

[StanFromIreland]

That worked, build is now around 3 minutes, and the tests seem to be running slightly faster than currently (down to 15 minutes from an average of 18).

[vstinner]

Ah, you add gh-157958: so I expected that you created an issue, but you added a reference to this PR instead. I would prefer an issue.

[vstinner]

Which branches are affected?

[vstinner]

If I check a recent GitHub Action job, I see:

• Sanitizers / UBSan: 37m 42s
• Sanitizers / TSan: 13m 49s
• Sanitizers / TSan (free-threading): 24m 32s

On this PR, I see:

• Tests / Sanitizers / UBSan (pull_request): 20m
• Tests / Sanitizers / TSan (pull_request): 13m
• Tests / Sanitizers / TSan (free-threading) (pull_request): 24m

So I see that the UBSan job is way faster: 37m 42s => 20m. The change works as expected.

[vstinner]

LGTM

[miss-islington-app]

Thanks @StanFromIreland for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

[miss-islington-app]

Thanks @StanFromIreland for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

[miss-islington-app]

Thanks @StanFromIreland for the PR 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

[miss-islington-app]

Sorry, @StanFromIreland, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker c57c8772b0b133c5580e94006e8b084f78753f49 3.13

[bedevere-app]

GH-158505 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-158506 is a backport of this pull request to the 3.15 branch.

[bedevere-app]

GH-158508 is a backport of this pull request to the 3.13 branch.