gh-75876: Run bigmem tests in a subprocess - #155302
Conversation
A test which really allocates the memory it asks for (that is, run with -M) now runs in a subprocess, so that the memory it uses and the address space it fragments are released when it ends. A dummy run stays in the process. The parent process watches the memory usage of the subprocess while waiting for it, so the separate watchdog process is no longer needed. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
serhiy-storchaka
added
needs backport to 3.13
|
Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15. |
|
Sorry, @serhiy-storchaka, I could not cleanly backport this to |
|
Sorry, @serhiy-storchaka, I could not cleanly backport this to |
|
Sorry, @serhiy-storchaka, I could not cleanly backport this to |
|
GH-156013 is a backport of this pull request to the 3.14 branch. |
|
GH-156014 is a backport of this pull request to the 3.13 branch. |
|
@serhiy-storchaka, |
|
Backporting to 3.15 needs merging several other backports first. |
A test which really allocates the memory it asks for (that is, run with -M) now runs in a subprocess, so that the memory it uses and the address space it fragments are released when it ends. A dummy run stays in the process. The separate watchdog process is no longer used. Unlike in the main branch, the parent process does not report the memory usage of the subprocess: get_process_memory_usage() (gh-150114) is not available on this branch. (cherry picked from commit 66d7c89)
2 participants
A test which really allocates the memory it asks for (that is, run with
-M) now runs in a subprocess, so that the memory it uses and the address space it fragments are released when it ends. A dummy run stays in the process.The parent process watches the memory usage of the subprocess while waiting for it, so
Lib/test/memory_watchdog.pyand the separate watchdog process are no longer needed._SubprocessTest.wait()calls back every second betweencommunicate()timeouts, which does not lose the subprocess output.Tested with
-M 4Gand-M 20Gon Linux, and with-M 4Gon FreeBSD and on a Windows debug build.