Skip to content

Interpreter repo's python symlink is absolute, so a repo-contents-cache hit points into another output base #4495

Closed as not planned

Description

@plx

Versions: rules_python 1.9.1 (the same line is in 2.5.0 and main; not run there), Bazel 8.4.2, macOS arm64, hermetic CPython 3.13.

What happens

python_repository ends with

rctx.delete("python")
rctx.symlink(python_bin, "python")

rctx.symlink writes an absolute target, so the repo's python entry is
<output_base>/external/rules_python++python+python_3_13_<platform>/bin/python3.

Since python_repository returns rctx.repo_metadata(reproducible = True) when a sha256 is set, Bazel moves the repo into the repo contents cache (--repo_contents_cache) and shares it between output bases. The absolute link goes with it. Every output base that gets the cache hit then has a python that resolves into the output base that first fetched the repo.

Two consequences, both observed:

  1. If that first output base no longer has the interpreter (it was deleted or expunged, or a fetch there was interrupted), every other output base fails in host_compatible_python_repo:

    rules_python:host_compatible_python_repo(@@rules_python++python+python_3_13_host) FAIL: repo.execute: CheckHostInterpreter: end: failure:
    src/main/tools/process-wrapper-legacy.cc:80: "execvp(<output_base>/external/rules_python++python+python_3_13_host/python, ...)": No such file or directory
    

    Nothing is re-extracted, because the cache entry is valid as far as Bazel can tell.

  2. If the first output base is later used with a different rules_python version that pins a different patch release, the other output bases silently run that interpreter as their host interpreter (we saw 3.13.9 where 3.13.13 was expected).

Reproduction

Two output bases A and B, one --repo_contents_cache directory, a workspace with a hermetic toolchain and any target that needs @python_3_13_host (a pip.parse hub is enough).

  1. In A: bazel build --nobuild <target>. readlink <cache>/<entry>/1/python prints an absolute path into A.
  2. Remove A (or interrupt a refetch of the interpreter repo there).
  3. In B: bazel build --nobuild <target> fails with the error above.

5 of 5 for us. With the contents cache disabled, step 3 passes. With a single output base it cannot happen, because the link leads back into the same output base.

A fix that works for us

Make the link relative on platforms that have ln:

     rctx.delete("python")
-    rctx.symlink(python_bin, "python")
+    if "windows" in platform:
+        rctx.symlink(python_bin, "python")
+    else:
+        repo_utils.execute_checked(
+            rctx,
+            op = "python_repository.SymlinkPython",
+            arguments = ["ln", "-s", python_bin, "python"],
+            logger = logger,
+        )

With it, 8 of 8 runs of the sequences above pass, no symlink in the cache points into an output base, and the host interpreter resolves into the cache entry. Windows is unchanged and untested. Another option would be for the repo not to declare itself reproducible while it contains an absolute link.

Activity

  1. aignas commented on Nov 13, 2026

    @aignas
    Collaborator

    I did look at it a little and it seems that bazel 9.0.1 and above may create the symlinks as relative: https://github.com/bazel-contrib/bazel_features/compare/v1.44.0...v1.45.0.patch

  2. aignas commented on Nov 14, 2026

    @aignas
    Collaborator

    Could you see if this is reproducible with 8.x of bazel? 8.4 is quite old.

  3. aignas commented on Nov 15, 2026

    @aignas
    Collaborator

    Could not reproduce:

    $ git diff
    diff --git a/MODULE.bazel b/MODULE.bazel
    index b7301c72..2a7baec5 100644
    --- a/MODULE.bazel
    +++ b/MODULE.bazel
    @@ -69,6 +69,7 @@ python.toolchain(
     use_repo(
         python,
         "python_3_14",
    +    "python_3_14_host",
         "pythons_hub",
         python = "python_versions",
     )
    
    $ USE_BAZEL_VERSION=8.x bazel build --repo_contents_cache=/home/aignas/tmp/rcc @python_3_14_host//:python
    
    # The host python symlink is absolute, as expected - it is not a reproducible repo
    $ ll /home/aignas/.cache/bazel/_bazel_aignas/6f0de8c9128ee8d5dbf27ba6dcc48bdd/external/+python+python_3_14_host/*
    .rwxr-xr-x 115 aignas 15 Nov 12:31 /home/aignas/.cache/bazel/_bazel_aignas/6f0de8c9128ee8d5dbf27ba6dcc48bdd/external/+python+python_3_14_host/BUILD.bazel
    lrwxrwxrwx   - aignas 15 Nov 12:31 /home/aignas/.cache/bazel/_bazel_aignas/6f0de8c9128ee8d5dbf27ba6dcc48bdd/external/+python+python_3_14_host/python -> /home/aignas/.cache/bazel/_bazel_aignas/6f0de8c9128ee8d5dbf27ba6dcc48bdd/external/+python+python_3_14_x86_64-unknown-linux-gnu/python
    .rw-r--r--   0 aignas 15 Nov 12:31 /home/aignas/.cache/bazel/_bazel_aignas/6f0de8c9128ee8d5dbf27ba6dcc48bdd/external/+python+python_3_14_host/REPO.bazel
    lrwxrwxrwx   - aignas 15 Nov 12:31 /home/aignas/.cache/bazel/_bazel_aignas/6f0de8c9128ee8d5dbf27ba6dcc48bdd/external/+python+python_3_14_host/STANDALONE_INTERPRETER -> /home/aignas/.cache/bazel/_bazel_aignas/6f0de8c9128ee8d5dbf27ba6dcc48bdd/external/+python+python_3_14_x86_64-unknown-linux-gnu/STANDALONE_INTERPRETER
    
    /home/aignas/.cache/bazel/_bazel_aignas/6f0de8c9128ee8d5dbf27ba6dcc48bdd/external/+python+python_3_14_host/bin:
    lrwxrwxrwx    - aignas 15 Nov 12:31 idle3 -> idle3.14
    .rwxrwxr-x  156 aignas  1 Jan  2024 idle3.14
    .rwxrwxr-x  234 aignas  1 Jan  2024 pip
    .rwxrwxr-x  234 aignas  1 Jan  2024 pip3
    .rwxrwxr-x  234 aignas  1 Jan  2024 pip3.14
    lrwxrwxrwx    - aignas 15 Nov 12:31 pydoc3 -> pydoc3.14
    .rwxrwxr-x  141 aignas  1 Jan  2024 pydoc3.14
    lrwxrwxrwx    - aignas 15 Nov 12:31 python -> python3.14
    lrwxrwxrwx    - aignas 15 Nov 12:31 python3 -> python3.14
    lrwxrwxrwx    - aignas 15 Nov 12:31 python3-config -> python3.14-config
    .rwxrwxr-x  79M aignas  1 Jan  2024 python3.14
    .rwxrwxr-x 3.8k aignas  1 Jan  2024 python3.14-config
    
    /home/aignas/.cache/bazel/_bazel_aignas/6f0de8c9128ee8d5dbf27ba6dcc48bdd/external/+python+python_3_14_host/include:
    drwxr-xr-x - aignas 15 Nov 12:31 python3.14
    
    /home/aignas/.cache/bazel/_bazel_aignas/6f0de8c9128ee8d5dbf27ba6dcc48bdd/external/+python+python_3_14_host/lib:
    drwxr-xr-x    - aignas 15 Nov 12:31 itcl4.3.8
    lrwxrwxrwx    - aignas 15 Nov 12:31 libpython3.14.so -> libpython3.14.so.1.0
    .rwxrwxr-x 137M aignas  1 Jan  2024 libpython3.14.so.1.0
    .rwxrwxr-x 8.6k aignas  1 Jan  2024 libpython3.so
    .rwxrwxr-x 2.4M aignas  1 Jan  2024 libtcl9.0.so
    .rwxrwxr-x 3.1M aignas  1 Jan  2024 libtcl9tk9.0.so
    drwxr-xr-x    - aignas 15 Nov 12:31 pkgconfig
    drwxr-xr-x    - aignas 15 Nov 12:31 python3.14
    drwxr-xr-x    - aignas 15 Nov 12:31 tcl9
    drwxr-xr-x    - aignas 15 Nov 12:31 tcl9.0
    drwxr-xr-x    - aignas 15 Nov 12:31 thread3.0.6
    drwxr-xr-x    - aignas 15 Nov 12:31 tk9.0
    
    /home/aignas/.cache/bazel/_bazel_aignas/6f0de8c9128ee8d5dbf27ba6dcc48bdd/external/+python+python_3_14_host/share:
    drwxr-xr-x - aignas 15 Nov 12:31 man
    
    # However, the specific reproducible python_repository repo is relative:
    $ ll /home/aignas/.cache/bazel/_bazel_aignas/6f0de8c9128ee8d5dbf27ba6dcc48bdd/external/+python+python_3_14_x86_64-unknown-linux-gnu/
    drwxr-xr-x   - aignas 15 Nov 12:31 bin
    .rwxr-xr-x 572 aignas 15 Nov 12:31 BUILD.bazel
    drwxr-xr-x   - aignas 15 Nov 12:31 include
    drwxr-xr-x   - aignas 15 Nov 12:31 lib
    lrwxrwxrwx   - aignas 15 Nov 12:31 python -> bin/python3
    .rw-r--r--   0 aignas 15 Nov 12:31 REPO.bazel
    drwxr-xr-x   - aignas 15 Nov 12:31 share
    .rwxr-xr-x 100 aignas 15 Nov 12:31 STANDALONE_INTERPRETER
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions