bazel: Every .bzl file must have a corresponding package, but X does not have one

Description of the problem / feature request:

Twice in the last two days when returning to rebuild a Bazel project, I’ve seen an error about it being unable to find the .bzl file for rules_cc rules. I think this has to do with the implicit loading of rules_cc by Bazel, as you’ll notice that nowhere is that file mentioned in the repository (https://sourcegraph.com/search?q=repo:^github\.com/google/llvm-bazel%24+“cc:defs.bzl”&patternType=regexp).

Bugs: what’s the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

As this is a non-deterministic thing, it’s a bit hard to reproduce. It seems to happen basically every time I fetch new commits in https://github.com/google/iree or https://github.com/google/llvm-bazel

What operating system are you running Bazel on?

Linux Debian

What’s the output of bazel info release?

Happens with Bazel 3.3.1 and 3.7.1

What’s the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

$ git remote get-url origin ; git rev-parse main ; git rev-parse HEAD
git@github.com:GMNGeoffrey/llvm-bazel.git
e463a75a963dc583ef1ccc6122aceaaabd5b8e09
e463a75a963dc583ef1ccc6122aceaaabd5b8e09

Have you found anything relevant by searching the web?

No

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 25 (25 by maintainers)

Most upvoted comments

Oh no… I think I know the problem… and it’s my fault… 🤦

@daily find /usr/local/google/home/gcmn/.cache/bazel* -mtime +12 -type f -delete

Bazel kept on using up all of my disk space, so I figured I could empty out old cache entries. It’s called “cache” so surely it must be ok to delete things from a correctness perspective. Apparently that’s not true 😁

But also

$ ll  $HOME/.cache/bazel/_bazel_gcmn/1655d0ec87ba8569e84422cbb0aeec46/external/bazel_skylib/BUILD
-r--r--r-- 1 gcmn primarygroup 1591 Dec 31  1999 /usr/local/google/home/gcmn/.cache/bazel/_bazel_gcmn/1655d0ec87ba8569e84422cbb0aeec46/external/bazel_skylib/BUILD

that doesn’t seem right?

So I did a bad thing for sure and this is my fault. I think there are some Bazel issues here that lead me to do this, in retrospect obviously bad, thing

  • Using tons of disk space. I’ve got a half terabyte drive. That’s not nothing. And Bazel was filling it up like ~weekly (?)
  • The name “cache” kind of implies that it’s not used for persistent storage ;-P

So deleted that cron job for now. I would love to come up with a safe replacement before Bazel fills up my drive again.

FTR I think I never would’ve figured this out without the suggestion of where to look for a BUILD file. The error about ‘@bazel_skylib//:workspace.bzl’ gave me nothing. It’s not any of the symlinks that are created in the source directory, which I did explore.

Also lol at all my guesses as to why I was seeing this “daily” and what that could correlate with. Turns out it just correlated with… days…

@GMNGeoffrey

Just add the strip_prefix will solve your problem.

And I think it is how bazel works indeed. If you don’t add strip_prefix, bazel will download the http_archive https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz to a folder named bazel-skylib-1.0.2. When you want to load something from that folder, you might have to do something like @bazel-skylib//bazel-skylib-1.0.2/....

What do you think ?

This happens to me just about every time I fetch new stuff from the repository at this point.

Hmmm what you describe has been my experience with other http_archive usage from GitHub releases. I believe I copied this from the instructions at https://github.com/bazelbuild/bazel-skylib/releases/tag/1.0.2 though and it looks identical. Also the error I experience here is non-deterministic, which isn’t the kind of error I’d expect here.

And testing it out, the Bazel skylib release archive does have different behavior here. I believe it does something fancy to avoid the need for the strip_prefix

gcmn@lt 2021-01-07 16:56 ~/Downloads/bazel-extract-test$ mv ../bazel-toolchains-3.3.1.tar.gz ./

gcmn@lt 2021-01-07 16:56 ~/Downloads/bazel-extract-test$ tar -xzf bazel-toolchains-3.3.1.tar.gz 

gcmn@lt 2021-01-07 16:56 ~/Downloads/bazel-extract-test$ ls
bazel-toolchains-3.3.1  bazel-toolchains-3.3.1.tar.gz

gcmn@lt 2021-01-07 16:56 ~/Downloads/bazel-extract-test$ rm -rf *

gcmn@lt 2021-01-07 16:57 ~/Downloads/bazel-extract-test$ mv ../bazel-skylib-1.0.2.tar.gz 
mv: missing destination file operand after '../bazel-skylib-1.0.2.tar.gz'
Try 'mv --help' for more information.

gcmn@lt 2021-01-07 16:57 ~/Downloads/bazel-extract-test$ mv ../bazel-skylib-1.0.2.tar.gz ./

gcmn@lt 2021-01-07 16:57 ~/Downloads/bazel-extract-test$ tar -xzf bazel-skylib-1.0.2.tar.gz 

gcmn@lt 2021-01-07 16:57 ~/Downloads/bazel-extract-test$ ls
bazel-skylib-1.0.2.tar.gz  bzl_library.bzl  CONTRIBUTORS       internal_setup.bzl  lib.bzl  rules                toolchains   workspace.bzl
BUILD                      CODEOWNERS       internal_deps.bzl  lib                 LICENSE  skylark_library.bzl  version.bzl

So note that the archive structure for bazel_skylib doesn’t have the nesting

Thanks. That then brings us to https://github.com/bazelbuild/buildtools/issues/923 😁 I also find I get this with rules_python

Oops it looks like I didn’t finish filling out the issue report. I am now running bazel clean --expunge multiple times a day for different repositories because of this issue