bazel: bazel 0.27 appears to no longer run on Trusty/RHEL/CentOS due to glibc compat

/usr/bin/bazel: relocation error: /usr/bin/bazel: symbol _ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE, version GLIBCXX_3.4.21 not defined in file libstdc++.so.6 with link time reference
make: *** [bazel_clean] Error 127

Is this expected or should Bazel still support these platforms?

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 43 (22 by maintainers)

Commits related to this issue

Teach cc_configure about BAZEL_LINKLIBS env variable Using this variable is will be possible to specify system libraries that should be added after default linker flags, after libraries to link, and ... — committed to bazelbuild/bazel by hlopko 5 years ago
Fix CI 14.04 is no longer supported by bazel https://github.com/bazelbuild/bazel/issues/8652 — committed to grailbio/bazel-compilation-database by keith 5 years ago
Teach cc_configure about BAZEL_LINKLIBS env variable Using this variable is will be possible to specify system libraries that should be added after default linker flags, after libraries to link, and ... — committed to siberex/bazel by hlopko 5 years ago
Teach cc_configure about BAZEL_LINKLIBS env variable Using this variable is will be possible to specify system libraries that should be added after default linker flags, after libraries to link, and ... — committed to irengrig/bazel by hlopko 5 years ago

Most upvoted comments

tl;dr I’ll fix this and will add official support for CentOS (we’ll have to see which versions of CentOS we can do, but 7.x and 8.x sound very likely - does anyone still need CentOS 6.x?).

@mzeren-vmw, could you share your patches (if any) / build command-line for Bazel that you use to statically link libstdc++? Maybe we can just use the same for our official releases.

It looks like the change is breaking more things than intended.

The change dropped official support for Ubuntu 14.04 LTS and instead made Ubuntu 16.04 LTS our main build platform to build Linux release artifacts on. It was unknown at the time which platforms that would break. The reasoning was that Ubuntu 14.04 LTS standard support ended, which makes it “de facto” end of life for the majority of users, because package repositories with security updates can now only be accessed via a paid subscription, which isn’t an option for our CI.

Users were asked on their opinion regarding this and we built several release candidates of Bazel 0.27.0 to learn about the larger impact of this change. I’m not aware of anyone complaining until after the final release.

Dropping support for Trusty before EOL seems not great but maybe understandable, […]

Yes. We have to strike a balance here and going with the standard support period of the most popular operating systems seems like a good option for now.

but I’m guessing that also dropping support for RHEL/CentOS is going to be very problematic for lots of people. Is there any chance of reconsidering this?

This is unfortunate and I’ll improve the current situation in two steps:

Build Linux releases in a way that let’s them work on supported RHEL / CentOS versions.
Make RHEL / CentOS a fully supported OS that we test Bazel on.

It would have been nice if people could have reported breakages earlier, while 0.27.0 was still in the release candidate phase (or even earlier, after the initial announcement that we’ll build Bazel on Ubuntu 16.04 LTS in the future, but I understand that it’s hard to extrapolate “it will stop working on CentOS 7.x” from that), but now let’s see that we can get this to work in 0.28.0 and hopefully in a 0.27.1 patch release.

philwo on Jun 17, 2019

FWIW our internal build of bazel links libstdc++ statically to resolve this type of issue.

+1 it would be awesome if the bazel official builds could do this also.

mattklein123 on Jun 17, 2019

Having multiple binaries would be unfortunate for users of tools like bazelbuild/bazelisk who happen to be on Centos7. I’m in this camp, and while I’ve built a forked bazel/bazelisk internally for my company in the past, I’d kinda like to avoid this in the future. Further, detecting the issue from bazelisk seems awkward too, although I suppose it could technically investigate the system libstdc++ to see if it has the correct symbols. Alternately, it could rely on the user setting an environment variable or a flag to pull in an alternately-built binary.

emidln on Jun 18, 2019

So, I’m not a big fan of linking against libstdc++ statically. What if we created additional released binaries for old libstdc++ in addition to our current binaries? I’m hopeful that this will work (didn’t try):

bazel build //src:bazel -c opt --copt=-D_GLIBCXX_USE_CXX11_ABI=0

Some background to why there is this libstdc++ incompatibility: https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html.

hlopko on Jun 18, 2019

@philwo FWIW, I think the static link fixes would also fix Trusty, at least in the short term, with the understanding it is not fully supported. We would really, really appreciate it if you could do this there also.

mattklein123 on Jun 17, 2019

Hmm, yikes. Dropping support for Trusty before EOL seems not great but maybe understandable, but I’m guessing that also dropping support for RHEL/CentOS is going to be very problematic for lots of people. Is there any chance of reconsidering this?

mattklein123 on Jun 17, 2019