feat(toolchain): wrap exec interpreter in launcher script for build actions - #4154
Draft
rickeylev wants to merge 6 commits into
Draft
feat(toolchain): wrap exec interpreter in launcher script for build actions#4154rickeylev wants to merge 6 commits into
rickeylev wants to merge 6 commits into
Conversation
…ctions Build actions invoking the execution-platform Python interpreter directly on bare binaries or raw symlinks fail under sandbox and remote execution environments because standard library modules (like encodings) cannot be found without a proper runfiles environment and PYTHONHOME. Wrap the interpreter in a launcher script: * Expand platform-specific launcher scripts (interpreter_tmpl.sh for POSIX sh and interpreter_tmpl.bat for Windows) in current_interpreter_executable. * Hermetically resolve the underlying interpreter through standard Bazel runfiles mechanisms (RUNFILES_DIR, RUNFILES_MANIFEST_FILE, and $0.runfiles). * Determine and export PYTHONHOME to ensure standard library modules are located within the runfiles tree. * Execute exec_tools.exec_interpreter[DefaultInfo].files_to_run in actions_run and py_console_script_gen. * Add analysis and action execution tests verifying sys.executable and stdlib modules reside in runfiles.
NixOS systems do not place sh in /bin/sh. Use /usr/bin/env sh so the launcher script can be executed on NixOS and other systems with non-standard /bin directories.
Rename template files from _tmpl to _template and encapsulate them within the exec_interpreter_template filegroup. * Rename interpreter_tmpl.sh to interpreter_template.sh and interpreter_tmpl.bat to interpreter_template.bat. * Remove the template files from exports_files in python/private/BUILD.bazel. * Point _template in interpreter_binary to //python/private:exec_interpreter_template. * Add comments explaining why PYTHONHOME points to the runfiles directory.
Note that RBE compatibility must still be verified in CI with the launcher script wrapper before removing the warning.
cmd.exe failed to parse interpreter_template.bat because of invalid stderr redirection and an unquoted parenthesis inside an if block. * Use 1>&2 for stderr redirection. * Replace literal parentheses with brackets in error message.
…e endings cmd.exe lacks Win32 long path awareness and fails with STATUS_NAME_TOO_LONG when resolving runfiles paths exceeding MAX_PATH in nested builds. Additionally, diff_test on Windows performs binary comparison (fc.exe /B), which failed when test_action.py wrote CRLF instead of LF. * Guard candidate runfiles paths in interpreter_template.bat to under 250 characters, falling back to the manifest when paths are too long. * Add current-directory fallback for relative manifest targets. * Use write_bytes with UTF-8 in test_action.py to ensure exact LF newlines across platforms.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Build actions that invoke the toolchain's execution-platform Python
interpreter directly fail under sandboxing or remote execution because
bare binaries and raw symlinks lack the necessary runfiles tree and
PYTHONHOME environment to locate standard library modules hermetically.
Wrap the interpreter in platform-specific launcher scripts for POSIX
and Windows that resolve the underlying interpreter through standard
Bazel runfiles lookup mechanisms and set PYTHONHOME to point to the
runfiles root. Wire these launchers into current_interpreter_executable
and interpreter_binary, and run them through actions_run and
py_console_script_gen.
Fixes #2703