bazel: Ventura 13.3 - bazel terminates abruptly

We have found a fix for this.

The firewall on Mac OS has changed behaviour since 13.3 we’ve tried to find details of this change but have been unable to since Apple haven’t published their changes for xnu for Mac OS 13.3 yet https://github.com/apple-oss-distributions/xnu/tags (xnu-8796.101.5 which is in Ventura 13.3 is missing and we haven’t found any release log for xnu)

We had set skip on lo0 to skip filtering on the loopback interface but this doesn’t seem to work as expected on Mac OS 13.3.

We have changed this to pass on lo0 all and added it after block all in our /etc/pf.conf and bazel seems to work correctly after this change.

Normally if you block lo0 bazel can’t connect to the blaze server after you start it - but in our previous configuration it seems to connect at the start and then it has issues connecting later on.

Some details on commands used to test various settings:

sudo pfctl -e #enable packet filtering
sudo pfctl -sr #view current status and rules
echo "block all \n pass on lo0 all" | sudo pfctl -f - # temporarily load some rules that block all outgoing traffic but allows all traffic on loopback interface
# sudo pfctl -f /etc/pf_custom.conf # if you want to load rules from a file

# git clone https://github.com/bazelbuild/bazel.git
cd bazel
bazelisk build '//:*'

#reset back to normal config and disable pfctl
sudo pfctl -F all #flush all rules
sudo pfctl -d #disable packet filtering

13.3.1 produces the same error.

We noticed that building the bazel project itself was also failing but was not failing on a laptop outside of the organisation with Ventura 13.3.

Some investigation with execsnoop led us to disabling pfctl.

After disabling pfctl builds started working.

We haven’t investigated why pfctl causes an issue yet but at least we are getting somewhere will post more information on why pfctl breaks things when we know more.

We’re seeing something similar, unclear yet exactly where the problem lies, but we have a user on macOS 13.3 reporting that analysis is stuck in an infinite loop. When analysis completes, it takes a lot longer and is constantly happening, even on no-ops. In a profile I see that when running a specific task for example, all external dependencies are constantly refetched and Bazel is doing a full re-analysis. This isn’t the case on the same project on a machine running macOS 13.2.1.

Hi @sgowroji, we have hundreds of users on Mac OS Ventura 13.2 and our large monorepo builds fine. As soon as users upgrade to 13.3 we get that error. I think it’s just after the Analysis phase that it crashes. Toy examples have failed to reproduce the issue. Just trying to build some large open source projects that hopefully might have the same issue.