bazel: Every .bzl file must have a corresponding package, but X does not have one

Description of the problem / feature request:

Twice in the last two days when returning to rebuild a Bazel project, I’ve seen an error about it being unable to find the .bzl file for rules_cc rules. I think this has to do with the implicit loading of rules_cc by Bazel, as you’ll notice that nowhere is that file mentioned in the repository (https://sourcegraph.com/search?q=repo:^github\.com/google/llvm-bazel%24+“cc:defs.bzl”&patternType=regexp).

Bugs: what’s the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

As this is a non-deterministic thing, it’s a bit hard to reproduce. It seems to happen basically every time I fetch new commits in https://github.com/google/iree or https://github.com/google/llvm-bazel

What operating system are you running Bazel on?

Linux Debian

What’s the output of `bazel info release`?

Happens with Bazel 3.3.1 and 3.7.1

What’s the output of `git remote get-url origin ; git rev-parse master ; git rev-parse HEAD` ?

$ git remote get-url origin ; git rev-parse main ; git rev-parse HEAD
git@github.com:GMNGeoffrey/llvm-bazel.git
e463a75a963dc583ef1ccc6122aceaaabd5b8e09
e463a75a963dc583ef1ccc6122aceaaabd5b8e09

Have you found anything relevant by searching the web?

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 25 (25 by maintainers)

Most upvoted comments

Oh no… I think I know the problem… and it’s my fault… 🤦

@daily find /usr/local/google/home/gcmn/.cache/bazel* -mtime +12 -type f -delete

Bazel kept on using up all of my disk space, so I figured I could empty out old cache entries. It’s called “cache” so surely it must be ok to delete things from a correctness perspective. Apparently that’s not true 😁

But also

$ ll  $HOME/.cache/bazel/_bazel_gcmn/1655d0ec87ba8569e84422cbb0aeec46/external/bazel_skylib/BUILD
-r--r--r-- 1 gcmn primarygroup 1591 Dec 31  1999 /usr/local/google/home/gcmn/.cache/bazel/_bazel_gcmn/1655d0ec87ba8569e84422cbb0aeec46/external/bazel_skylib/BUILD

that doesn’t seem right?

So I did a bad thing for sure and this is my fault. I think there are some Bazel issues here that lead me to do this, in retrospect obviously bad, thing

Using tons of disk space. I’ve got a half terabyte drive. That’s not nothing. And Bazel was filling it up like ~weekly (?)
The name “cache” kind of implies that it’s not used for persistent storage ;-P

So deleted that cron job for now. I would love to come up with a safe replacement before Bazel fills up my drive again.

FTR I think I never would’ve figured this out without the suggestion of where to look for a BUILD file. The error about ‘@bazel_skylib//:workspace.bzl’ gave me nothing. It’s not any of the symlinks that are created in the source directory, which I did explore.

Also lol at all my guesses as to why I was seeing this “daily” and what that could correlate with. Turns out it just correlated with… days…

GMNGeoffrey on Apr 8, 2021

@GMNGeoffrey

Just add the strip_prefix will solve your problem.

And I think it is how bazel works indeed. If you don’t add strip_prefix, bazel will download the http_archive https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz to a folder named bazel-skylib-1.0.2. When you want to load something from that folder, you might have to do something like @bazel-skylib//bazel-skylib-1.0.2/....

What do you think ?

vnghia on Jan 7, 2021

This happens to me just about every time I fetch new stuff from the repository at this point.

GMNGeoffrey on Dec 15, 2020

Hmmm what you describe has been my experience with other http_archive usage from GitHub releases. I believe I copied this from the instructions at https://github.com/bazelbuild/bazel-skylib/releases/tag/1.0.2 though and it looks identical. Also the error I experience here is non-deterministic, which isn’t the kind of error I’d expect here.

And testing it out, the Bazel skylib release archive does have different behavior here. I believe it does something fancy to avoid the need for the strip_prefix

gcmn@lt 2021-01-07 16:56 ~/Downloads/bazel-extract-test$ mv ../bazel-toolchains-3.3.1.tar.gz ./

gcmn@lt 2021-01-07 16:56 ~/Downloads/bazel-extract-test$ tar -xzf bazel-toolchains-3.3.1.tar.gz 

gcmn@lt 2021-01-07 16:56 ~/Downloads/bazel-extract-test$ ls
bazel-toolchains-3.3.1  bazel-toolchains-3.3.1.tar.gz

gcmn@lt 2021-01-07 16:56 ~/Downloads/bazel-extract-test$ rm -rf *

gcmn@lt 2021-01-07 16:57 ~/Downloads/bazel-extract-test$ mv ../bazel-skylib-1.0.2.tar.gz 
mv: missing destination file operand after '../bazel-skylib-1.0.2.tar.gz'
Try 'mv --help' for more information.

gcmn@lt 2021-01-07 16:57 ~/Downloads/bazel-extract-test$ mv ../bazel-skylib-1.0.2.tar.gz ./

gcmn@lt 2021-01-07 16:57 ~/Downloads/bazel-extract-test$ tar -xzf bazel-skylib-1.0.2.tar.gz 

gcmn@lt 2021-01-07 16:57 ~/Downloads/bazel-extract-test$ ls
bazel-skylib-1.0.2.tar.gz  bzl_library.bzl  CONTRIBUTORS       internal_setup.bzl  lib.bzl  rules                toolchains   workspace.bzl
BUILD                      CODEOWNERS       internal_deps.bzl  lib                 LICENSE  skylark_library.bzl  version.bzl

So note that the archive structure for bazel_skylib doesn’t have the nesting

GMNGeoffrey on Jan 8, 2021

Thanks. That then brings us to https://github.com/bazelbuild/buildtools/issues/923 😁 I also find I get this with rules_python

GMNGeoffrey on Dec 31, 2020

Oops it looks like I didn’t finish filling out the issue report. I am now running bazel clean --expunge multiple times a day for different repositories because of this issue

GMNGeoffrey on Dec 30, 2020