bazel: Unexpected interaction between toolchain resolution and bzlmod
Description of the bug:
When bzlmod is enabled (as it is the case in Bazel 7.0.0), toolchain registered by default can unexpectedly shadow toolchains declared in the WORKSPACE
file. This makes the oucome of toolchain resolution dependent on whether bzlmod is enabled, even if the MODULE.bazel
file of the main repository is empty (as it is by default)
The root cause seems to be different from #17289 : there, AFAIU that one is caused by a subtle interaction between label mapping and toolchain resolution, here it’s much more straightforward.
Some debugging indicates that what happens is that RegisteredToolchainsFunction
creates the list of registered toolchains by concatenating two lists: those from bzlmod, then those from the WORKSPACE file. rules_python
is by default and immutably (except if bzlmod is disabled) a bzlmod module and it registers @bazel_tools//tools/python:_autodetecting_py_runtime_pair
as a Python toolchain. Since that toolchain doesn’t have any exec or target constraints, it seems to win toolchain selection by default, regardless of what other toolchains come after.
Therefore, no matter what is written in the WORKSPACE
file, one gets the autodetecting Python toolchain.
I’m not sure if this is WAI; it could be construed as such, but it sure is surprising and a hindrance to migration to bzlmod.
Which category does this issue belong to?
No response
What’s the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
cat > WORKSPACE <<'EOF'
register_toolchains("//tc:py_toolchain")
EOF
cat > BUILD <<'EOF'
py_binary(
name = "py",
srcs = ["py.py"],
)
EOF
cat > py.py <<'EOF'
print("Hello, Python!")
EOF
mkdir tc
cat > tc/BUILD <<'EOF'
load("@bazel_tools//tools/python:toolchain.bzl", "py_runtime_pair")
py_runtime(
name = "python2",
files = ["ok.py2"],
interpreter = "ok.py2",
python_version = "PY2",
)
py_runtime(
name = "python3",
files = ["ok.py3"],
interpreter = "ok.py3",
python_version = "PY3",
)
py_runtime_pair(
name = "pair",
py2_runtime = ":python2",
py3_runtime = ":python3",
)
toolchain(
name = "py_toolchain",
toolchain = ":pair",
toolchain_type = "@bazel_tools//tools/python:toolchain_type",
)
EOF
bazel run --enable_bzlmod :py # BUG: Prints "Hello, Python!", ignoring the toolchain
bazel run --noenable_bzlmod :py # WORKS: Fails to find the dummy interpreter defined above
Which operating system are you running Bazel on?
Linux
What is the output of bazel info release
?
development version
If bazel info release
returns development version
or (@non-git)
, tell us how you built Bazel.
From sources
What’s the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD
?
4a29f0851d1cde0240793cdc7a2e2cab926d31b7
Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.
Pretty obviously when --enable_bzlmod
was flipped to true.
Have you found anything relevant by searching the web?
There are a number of pertinent bugs:
- https://github.com/bazelbuild/bazel/issues/17289 (but that has a different root cause, if I understand correctly)
- https://github.com/bazelbuild/rules_python/issues/1313 (seems to be very similar, but I haven’t seen evidence that the bzlmod folks looked at this and has less information as to why this happens)
Any other information, logs, or outputs that you want to share?
No response
About this issue
- Original URL
- State: closed
- Created 7 months ago
- Comments: 20 (17 by maintainers)
Commits related to this issue
- Ensure that the autodetected python runtimes are marked as compatible with the host platform. This should prevent them from being used inappropriately in remote execution. Part of changes for #20354... — committed to katre/bazel by katre 7 months ago
- Give `WORKSPACE` toolchains and platforms precedence over non-root modules RELNOTES[INC]: Toolchains and execution platforms are now registered in the following order with `--enable_bzlmod`: 1. root ... — committed to iancha1992/bazel by fmeum 7 months ago
- [7.0.0] Give `WORKSPACE` toolchains and platforms precedence over non-root mo… (#20430) …dules RELNOTES[INC]: Toolchains and execution platforms are now registered in the following order with `--... — committed to bazelbuild/bazel by iancha1992 7 months ago
- Release 7.0.0 (2023-12-11) Baseline: d60ce2c7c86393638c77698c00c2168a7a936a53 Release Notes: + **[Incompatible]** This has the side effect of changing the message on unsuccessful builds from ``` F... — committed to bazelbuild/bazel by a-googler 7 months ago
- Release 7.0.0 (2023-12-11) Baseline: d60ce2c7c86393638c77698c00c2168a7a936a53 Release Notes: + **[Incompatible]** This has the side effect of changing the message on unsuccessful builds from ``` F... — committed to uber-common/bazel by a-googler 7 months ago
This is a fixable problem, actually. We could annotate toolchains registered in WORKSPACE suffixes with a “low priority” bit (messy, but technically possible), then order the toolchains per 1) root MODULE.bazel 2) WORKSPACE sans suffix 3) non-root Bzlmod stuff 4) WORKSPACE suffix. This would address the Kleaf use case.
@bazel-io fork 7.0.0
Whoops, sorry
One possible solution I see is to change the order of registration to the following in order of decreasing precedence:
Independent of that particular issue, the auto-detecting Python toolchain should have constraints to match on the auto-detected host platform, otherwise it probably breaks remote execution of Python tools.