Fix embedded Python executable on Windows 3.10 - #820
Open
MilesCranmerBot wants to merge 6 commits into
Open
MilesCranmerBot wants to merge 6 commits into
MilesCranmerBot wants to merge 6 commits into
Conversation
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
Member
|
Good PR but Python 3.10 reaches EOL in a month (https://devguide.python.org/versions/) at which point PythonCall will also drop support for it. So I'm inclined to leave this buggy behaviour and just advise downstream users to require Python >=3.11. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Windows CPython 3.10 derives
sys.executablefrom the host process even afterPy_SetProgramName. When Julia embeds Python, that leavessys.executablepointing atjulia.exe, so Python subprocess launches use the wrong program.Set the executable path before
Py_InitializeExusing CPython 3.10's exported_Py_SetProgramFullPath. The private API call is restricted to Windows Python 3.10; other versions and already-initialized interpreters keep their existing initialization path.The regression starts a fresh Julia process with the configuration inherited from JuliaCall, checks its executable against the parent Python, and launches a Python subprocess through it. The test does not set the interpreter environment variable. Existing tests are unchanged.
Verification
Before the fix, the regression failed on Windows Python 3.10 with both official CPython and Conda. Windows Python 3.14 and Ubuntu Python 3.10 passed.
With the fix, all four configurations pass.
This addresses the remaining worker-initialization failure exposed by astroautomata/PySR#1366.
The full upstream Tests workflow also passed all 16 Julia/Python jobs. Benchmark execution completed, but its job failed while posting the result comment with HTTP 403; the reporting fix is tracked separately in #819.
PySR integration
The controlled Windows comparison tests the released dependencies before and after PySR #1362, a JuliaCall 0.9.34 control, and this upstream pair with the PySR compatibility assignment removed.
Before #1362, a Python 3.10 parent gets a separate CondaPkg Python 3.14 worker. After #1362, the worker uses Python 3.10 but its
sys.executablelaunches Julia and fails withERROR: unknown option `-c`. The 0.9.34 control reproduces the same Windows 3.10 initialization defect.With this source build and no PySR compatibility assignment, both import orders pass the unchanged worker regression and launch Python successfully. The complete
python -m pysr test main,startupsuites pass on both Windows Conda configurations:The original startup-test file has the same SHA-256 in every configuration. No PySR tests or assertions were changed.
A coordinated release containing #814 and this fix is required before PySR can remove its downstream workaround and require the corrected dependency. This validation uses upstream source builds; the published-package canary continues to test released packages.