bazel: incompatible_disallow_empty_glob: fail if a glob doesn't match anything
The glob() function tends to be error-prone, because any typo in a path will silently return an empty list. This has been a source of confusion and bugs for many users.
We’ve just added a new argument to glob: allow_empty. For example, let’s say your code looks like:
glob([pattern_a, pattern_b], exclude = [pattern_c], allow_empty = False)
This code will fail if pattern_a or pattern_b doesn’t match anything. It also fails if the whole function (after excluding pattern_c) doesn’t match anything. We believe this behavior is safer in general.
In rare cases, this is intentional that a pattern doesn’t match anything, e.g. when a source file is optional. When it happens, users should tell their intent explicitly by setting allow_empty = True.
-
What’s going to change? The default value of
allow_emptyused to be True. It will change to False. -
How to update your code? If a glob fails, use
allow_empty = Trueto keep the old behavior.
If you enable the flag, make sure you are using Bazel 0.29 or above (a bug in Bazel 0.28 can prevent you from testing your code).
About this issue
- Original URL
- State: open
- Created 5 years ago
- Comments: 20 (17 by maintainers)
Commits related to this issue
- Introduce flag --incompatible_disallow_empty_glob=true Implements https://github.com/bazelbuild/bazel/issues/8195 When the flag is enabled, the default value of allow_empty in glob is False. RELNOT... — committed to bazelbuild/bazel by laurentlb 5 years ago
- Release 0.27.0 (2019-06-17) Baseline: 5935259724bebd1c4bdebc90e159d0f655c8c219 Cherry picks: + fe81b49e727efdcc90a270520af193af75f4e31d: Support of using the absolute path profile for LLVM'... — committed to bazelbuild/bazel by a-googler 5 years ago
- Introduce flag --incompatible_disallow_empty_glob=true Implements https://github.com/bazelbuild/bazel/issues/8195 When the flag is enabled, the default value of allow_empty in glob is False. RELNOT... — committed to irengrig/bazel by laurentlb 5 years ago
- Release 0.27.0 (2019-06-17) Baseline: 5935259724bebd1c4bdebc90e159d0f655c8c219 Cherry picks: + fe81b49e727efdcc90a270520af193af75f4e31d: Support of using the absolute path profile for LLVM'... — committed to irengrig/bazel by a-googler 5 years ago
- Release 0.27.0 (2019-06-17) Baseline: 5935259724bebd1c4bdebc90e159d0f655c8c219 Cherry picks: + fe81b49e727efdcc90a270520af193af75f4e31d: Support of using the absolute path profile for LLVM'... — committed to buchgr/bazel by a-googler 5 years ago
- Release 0.27.0 (2019-06-17) Baseline: 5935259724bebd1c4bdebc90e159d0f655c8c219 Cherry picks: + fe81b49e727efdcc90a270520af193af75f4e31d: Support of using the absolute path profile for LLVM'... — committed to siberex/bazel by a-googler 5 years ago
- Introduce flag --incompatible_disallow_empty_glob=true Implements https://github.com/bazelbuild/bazel/issues/8195 When the flag is enabled, the default value of allow_empty in glob is False. RELNOT... — committed to irengrig/bazel by laurentlb 5 years ago
- Release 0.27.0 (2019-06-17) Baseline: 5935259724bebd1c4bdebc90e159d0f655c8c219 Cherry picks: + fe81b49e727efdcc90a270520af193af75f4e31d: Support of using the absolute path profile for LLVM'... — committed to irengrig/bazel by a-googler 5 years ago
- Fix errors with incompatible_disallow_empty_glob Allow py_library sources to be empty. If --incompatible_disallow_empty_glob is set then generated py_library targets will fail if there are no .py fil... — committed to gergelyfabian/rules_python by gergelyfabian 4 years ago
- Fix errors with incompatible_disallow_empty_glob Allow py_library sources to be empty. If --incompatible_disallow_empty_glob is set then generated py_library targets will fail if there are no .py fil... — committed to gergelyfabian/rules_python by gergelyfabian 4 years ago
- Fix errors with incompatible_disallow_empty_glob Allow py_library sources to be empty. If --incompatible_disallow_empty_glob is set then generated py_library targets will fail if there are no .py fil... — committed to gergelyfabian/rules_python_external by gergelyfabian 4 years ago
- Fix errors with incompatible_disallow_empty_glob Allow py_library sources to be empty. If --incompatible_disallow_empty_glob is set then generated py_library targets will fail if there are no .py fil... — committed to gergelyfabian/rules_python by gergelyfabian 4 years ago
- Fix errors with incompatible_disallow_empty_glob Allow py_library sources to be empty. If --incompatible_disallow_empty_glob is set then generated py_library targets will fail if there are no .py fil... — committed to gergelyfabian/rules_python by gergelyfabian 4 years ago
- Fix errors with incompatible_disallow_empty_glob (#315) Allow py_library sources to be empty. If --incompatible_disallow_empty_glob is set then generated py_library targets will fail if there are n... — committed to bazelbuild/rules_python by gergelyfabian 4 years ago
- Mention `--incompatible_disallow_empty_glob` in the error message. After a flag flip, some users could be surprised by the error. Mentioning the flag name will let them disable it or easily find info... — committed to bazelbuild/bazel by laurentlb 4 years ago
- keep file permissions when unpacking add zipextended.py to BUILD fix import path use pip internal unzip use pip internal unzip tools remove custom zipfile implementation Add documentation of com... — committed to gmweaver/rules_python by deleted user 4 years ago
- keep file permissions when unpacking add zipextended.py to BUILD fix import path use pip internal unzip use pip internal unzip tools remove custom zipfile implementation Add documentation of com... — committed to gmweaver/rules_python by deleted user 4 years ago
- Remove none existing folders This folders do not exist and this prevents to build with --incompatible_disallow_empty_glob https://github.com/bazelbuild/bazel/issues/8195 — committed to limdor/upb by limdor 2 years ago
- Be explicit about alowing empty globs There are several globs that are empty and this prevents building with the incompatible flag #8195. This commit just makes it explicit that empty is allowed. — committed to limdor/bazel by limdor 2 years ago
- Be explicit about alowing empty globs (second part) There are several globs that are empty and this prevents building with the incompatible flag #8195. This commit just makes it explicit that empty i... — committed to limdor/bazel by limdor 2 years ago
Ok understood, I will take a look to the failures then. In general the fixes are quite trivial, at least to get the default behavior. The problem is getting reviews in timely manner. The problem that I might be facing in downstream projects is that setting the allow_empty = True might not be well received and some projects might want to investigate why that is the case. However my rationale here is that first it should be set allow_empty to True and then investigate, with this way we can flip the default behavior and stop introducing cases where the glob is empty but the developer did not expect that.
I understand that the downstream projects are important but at some point there is the whole Bazel comunity that is suffering from not flipping this flag.
My proposal would be to try to get the PRs for the downstream projects where it sets allow_empty to true for the affected globs, and in the hypothetical case where they don’t want to merge it, consider flipping it anyway. But somehow I’m sure we should be able to convince the maintainers of the downstream projects that setting the default explicitly is better than doing nothing.
Update: All PRs in Bazel have been merged except the flag flip itself. The remaining part is the downstream projects that are tracked here https://github.com/bazelbuild/bazel/pull/15327
@bazel-io flag
@meteorcloudy I believe migrating this flag is a chicken and egg problem. Enabling this does not really solve a problem for users. It merely establishes a sane default for globbing. Imho donwstream projects will most likely not adapt until there is an incentive.
What I am going for is, imho we should flip this flag soon (maybe Bazel 6.0?). This might be the gentle push for downstream projects to adapt to the flag. If they don’t want to, setting the old behavior via the incompatibility flag is easy enough.