bazel: Runfiles workspace directories use wrong workspace name if repository renaming is in effect
Description of the problem / feature request:
If a workspace B depends on a workspace A, but uses the name C instead of A to name the workspace, the runfiles for workspace A will appear as C in the runfiles directory, breaking runfiles lookup for any library in workspace A.
Bugs: what’s the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
-
/tmp/test/a/WORKSPACE:
workspace(name = "a")
-
/tmp/test/a/BUILD:
cc_library( name = "lib", srcs = ["lib.cc"], hdrs = ["lib.h"], data = ["data.txt"], visibility = ["//visibility:public"], deps = ["@bazel_tools//tools/cpp/runfiles"], )
-
/tmp/test/a/lib.h:
#include <string> void Test(const std::string& argv0);
-
/tmp/test/a/lib.cc:
#include "lib.h" #include <exception> #include <fstream> #include <iostream> #include <string> #include "tools/cpp/runfiles/runfiles.h" void Test(const std::string& argv0) { using bazel::tools::cpp::runfiles::Runfiles; const Runfiles* runfiles = Runfiles::Create(argv0); if (runfiles == nullptr) throw std::runtime_error("missing runfiles"); std::string filename = runfiles->Rlocation("a/data.txt"); if (filename.empty()) throw std::runtime_error("can't find runfile"); std::ifstream stream(filename); stream.exceptions(std::ios::failbit | std::ios::badbit); if (!stream.is_open()) throw std::runtime_error("can't open runfile"); std::cout.exceptions(std::ios::failbit | std::ios::badbit); std::cout << stream.rdbuf() << std::endl; }
-
/tmp/test/a/data.txt:
hello world
-
/tmp/test/b/WORKSPACE:
workspace(name = "b") local_repository( name = "c", ## "a" path = "/tmp/test/a", repo_mapping = {"@c": "@a"}, )
-
/tmp/test/b/BUILD:
cc_binary( name = "bin", srcs = ["bin.cc"], deps = ["@c//:lib"], ## "@a//:lib" )
-
/tmp/test/b/bin.cc:
#include "lib.h" int main(int argc, char** argv) { Test(argc > 0 ? argv[0] : ""); }
Then, cd /tmp/test/b && bazel run :bin
will fail. In /tmp/test/b/bazel-bin/bin.runfiles there will be a subdirectory c
(i.e. the name of the repository in b
), but no subdirectory a
(i.e. the actual name of the a
repository).
The same works after renaming the local repository in b/WORKSPACE
to a
(as the comments indicate) and commenting out the repo_mapping
attribute.
This is a problem because code in workspace A can’t possible know that its clients import it under a different name. Therefore, code in workspace A can only refer to runfiles within its own workspace under the name that it has declared in its own WORKSPACE file (a
in this example). This means that the runfiles actually have to be available under the a
name, independent of any repository mappings. So Bazel should not only create a symlink c/data.txt
in the runfiles directory, but also a/data.txt
. More generally, any runfile should be available under both its local workspace name and any renamed/remapped workspace names. In case of name clashes, Bazel should signal an error, as it does for other name clashes.
What operating system are you running Bazel on?
macOS
What’s the output of bazel info release
?
release 5.0.0-homebrew
What’s the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD
?
Irrelevant
Have you found anything relevant by searching the web?
There are probably many related discussions, but I haven’t found a bug report about this specific issue itself. Probably the problem will become much worse with bzlmod due to its extensive use of repository renaming, and I’d consider it a blocker for enabling bzlmod by default.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 39 (26 by maintainers)
I don’t actually see the two proposals as being exclusive and if we are willing to do the work, we could implement and benefit from both:
If we get the order of lookups in the Bazel-provided runfiles libraries right, they should remain drop-in compatible with a hypothetical
--bzlmod_compatibility_runfiles
flag that expands the runfiles tree in the way @comius proposes. Since the flag would be controlled by end users, rulesets would still be incentivized to migrate to the official runfiles libraries, ensuring optimal (future) efficiency and cross-platform support. At the same time, end users with complex projects could use--bzlmod_compatibility_runfiles
as an escape hatch to keep their existing runfiles lookup routines functioning as well as possible while consciously accepting the performance price.(Now that I am reading through this thread again, this is actually quite similar to https://github.com/bazelbuild/bazel/issues/15029#issuecomment-1072112309)
Given that this functionality would be mostly relevant for end users rather than rulesets (@comius, please correct me here if I should be misrepresenting your proposal), it wouldn’t be an issue if we would introduce this flag only in 6.1.0, giving us more time to think this through and implement it.
@comius What do you think, could that be a reasonable path forward? If you are interested, we could schedule a short call and go over how this could be implemented and whether there are any unforeseen issues.
May I formally request that changes to runfiles library to go through the proposal process in https://github.com/bazelbuild/proposals? It might help tremendously for future references looking back as well as lay a good foundation for future documentation effort. (there is no doc for runfiles lib today)
This is indeed a very important problem to solve, @Wyverald can you schedule a meeting and add interested people to join the brainstorming?