Fix Linux pytest shutdown with explicit Julia signal handling - #818
Open
MilesCranmerBot wants to merge 4 commits into
Open
Fix Linux pytest shutdown with explicit Julia signal handling#818MilesCranmerBot wants to merge 4 commits into
MilesCranmerBot wants to merge 4 commits into
Conversation
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
Contributor
|
Can confirm I believe this is a valid fix to some of the issues I've been seeing in PySR CI jobs. Seems like pytest and juliacall are fighting at shutdown for some reason. Not sure if this is also fixes #764 but perhaps it might. |
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
Member
|
This feels like quite a magical/intrusive PR. I'm not totally opposed to it but is there any reason the user cannot just do |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On Linux, pytest's
faulthandlerplugin can replace Julia's signal handlers during teardown. In a PySR notebook reproduction, all seven cells passed, then the pytest subprocess exited with SIGSEGV insidejl_atexit_hook.A native debugger trace showed the ownership sequence: CPython installed its SIGSEGV handler, Julia installed its safepoint handler, then
faulthandler.disable()restored CPython's savedSIG_DFL. A Julia worker subsequently faulted on a GC safepoint page during shutdown. The shutdown path is unchanged between v0.9.35 and the upstream base of this PR.Change
Add a small pytest entry-point plugin that blocks pytest's
faulthandlerplugin before configuration when both conditions hold:yes, using-X juliacall-handle-signalsbeforePYTHON_JULIACALL_HANDLE_SIGNALS.The entry point lives outside the
juliacallpackage so pytest startup does not initialize Julia. Merely installing JuliaCall does not activate the policy. A warning explains the diagnostic tradeoff, and-p no:juliacallopts out.Pytest-managed Python fatal-error and timeout tracebacks are unavailable under this policy. A positive
faulthandler_timeoutcombined withfaulthandler_exit_on_timeoutraises a configuration error instead of silently removing timeout enforcement. Malformed boolean configuration still fails. Pytest versions without the exit-on-timeout option remain supported.This only coordinates pytest's diagnostic plugin. Arbitrary
faulthandler.disable()calls or native handler replacement while Julia is running can still disrupt Julia's signals. Julia fatal diagnostics remain available.Related to #764 and #763. This PR does not bypass
jl_atexit_hook, Julia finalizers or user atexit hooks, and does not claim to fix every possible cause of #764.Verification
Tested a built and installed wheel on Linux x86-64 with Python 3.13.11, Julia 1.12.7 and SymbolicRegression 2.4.1:
PYTEST_ADDOPTSor a-p no:faulthandlerworkaround.main,cli,startup,interruptsuite: 204 passed. Four independently executed shards each passed 51 tests. The unchanged preservation report confirmed all original test IDs and skips were preserved and the shards partitioned the full suite exactly.The configuration regression failed on the published wheel before the integration. Final campaigns ran sequentially against a separately provisioned project using documented executable/project overrides. Unchanged PySR tests added their optional dependencies; the developed PythonCall path/version and SymbolicRegression version stayed unchanged.
To exercise the installed-wheel regression from this checkout:
With PySR and its development dependencies installed in the verification environment, the unchanged consumer commands were:
The final campaign used six Julia threads and Julia signal handling enabled. No PySR source, assertion, skip, search budget or test timeout was changed by this PR.