bazel: TensorFlow Cython extension fails to link on Mac with Bazel 0.23.0 but not with Bazel 0.22.0.

Description of the problem / feature request:

TensorFlow Cython extension fails to link on Mac with Bazel 0.23.0 but not with Bazel 0.22.0.

Bugs: what’s the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Repro, may require installing TF’s build deps first.

$ git clone https://github.com/tensorflow/tensorflow.git
$ cd tensorflow
$ yes '' | ./configure  # Choose default options.
$ bazel build tensorflow/python:framework_fast_tensor_util

With bazel 0.23.0, the build action seems to be:

  external/local_config_cc/cc_wrapper.sh -lc++ -fobjc-link-runtime -shared -o bazel-out/darwin-opt/bin/tensorflow/python/framework/fast_tensor_util.so -Wl,-force_load,bazel-out/darwin-opt/bin/tensorflow/python/_objs/framework/fast_tensor_util.so/fast_tensor_util.o -headerpad_max_install_names -no-canonical-prefixes '-mmacosx-version-min=10.14')

which produces

...
INFO: From Compiling tensorflow/python/framework/fast_tensor_util.cpp:
In file included from bazel-out/darwin-opt/genfiles/tensorflow/python/framework/fast_tensor_util.cpp:581:
In file included from bazel-out/darwin-opt/genfiles/external/local_config_python/numpy_include/numpy/arrayobject.h:4:
In file included from bazel-out/darwin-opt/genfiles/external/local_config_python/numpy_include/numpy/ndarrayobject.h:12:
In file included from bazel-out/darwin-opt/genfiles/external/local_config_python/numpy_include/numpy/ndarraytypes.h:1822:
bazel-out/darwin-opt/genfiles/external/local_config_python/numpy_include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with "          "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
#warning "Using deprecated NumPy API, disable it with " \
 ^
1 warning generated.
ERROR: /Users/phawkins/p/jax/tensorflow/tensorflow/python/BUILD:6155:1: Linking of rule '//tensorflow/python:framework/fast_tensor_util.so' failed (Exit 1)
Undefined symbols for architecture x86_64:
  "_PyBaseObject_Type", referenced from:
      __pyx_pw_10tensorflow_6python_9framework_16fast_tensor_util_1AppendBFloat16ArrayToTensorProto(_object*, _object*, _object*) in fast_tensor_util.o
      __pyx_pw_10tensorflow_6python_9framework_16fast_tensor_util_3AppendFloat16ArrayToTensorProto(_object*, _object*, _object*) in fast_tensor_util.o
      __pyx_pw_10tensorflow_6python_9framework_16fast_tensor_util_5AppendFloat32ArrayToTensorProto(_object*, _object*, _object*) in fast_tensor_util.o
      __pyx_pw_10tensorflow_6python_9framework_16fast_tensor_util_7AppendFloat64ArrayToTensorProto(_object*, _object*, _object*) in fast_tensor_util.o
      __pyx_pw_10tensorflow_6python_9framework_16fast_tensor_util_9AppendInt32ArrayToTensorProto(_object*, _object*, _object*) in fast_tensor_util.o
      __pyx_pw_10tensorflow_6python_9framework_16fast_tensor_util_11AppendUInt32ArrayToTensorProto(_object*, _object*, _object*) in fast_tensor_util.o
      __pyx_pw_10tensorflow_6python_9framework_16fast_tensor_util_13AppendInt64ArrayToTensorProto(_object*, _object*, _object*) in fast_tensor_util.o
      ...
  "_PyBuffer_Release", referenced from:
      __pyx_pw_10tensorflow_6python_9framework_16fast_tensor_util_1AppendBFloat16ArrayToTensorProto(_object*, _object*, _object*) in fast_tensor_util.o
      __Pyx__GetBufferAndValidate(bufferinfo*, _object*, __Pyx_TypeInfo*, int, int, int, __Pyx_BufFmt_StackElem*) in fast_tensor_util.o
      __pyx_pw_10tensorflow_6python_9framework_16fast_tensor_util_3AppendFloat16ArrayToTensorProto(_object*, _object*, _object*) in fast_tensor_util.o
      __pyx_pw_10tensorflow_6python_9framework_16fast_tensor_util_5AppendFloat32ArrayToTensorProto(_object*, _object*, _object*) in fast_tensor_util.o
      __pyx_pw_10tensorflow_6python_9framework_16fast_tensor_util_7AppendFloat64ArrayToTensorProto(_object*, _object*, _object*) in fast_tensor_util.o
      __pyx_pw_10tensorflow_6python_9framework_16fast_tensor_util_9AppendInt32ArrayToTensorProto(_object*, _object*, _object*) in fast_tensor_util.o
      __pyx_pw_10tensorflow_6python_9framework_16fast_tensor_util_11AppendUInt32ArrayToTensorProto(_object*, _object*, _object*) in fast_tensor_util.o
      ...
  "_PyBytes_FromStringAndSize", referenced from:
      _PyInit_fast_tensor_util in fast_tensor_util.o
  "_PyCFunction_NewEx", referenced from:
      _PyInit_fast_tensor_util in fast_tensor_util.o
  "_PyCFunction_Type", referenced from:
      __Pyx_PyObject_Append(_object*, _object*) in fast_tensor_util.o
      __Pyx_PyObject_CallOneArg(_object*, _object*) in fast_tensor_util.o
...

On bazel 0.22.0, the corresponding build action is:

  external/local_config_cc/cc_wrapper.sh -lc++ -fobjc-link-runtime -shared -o bazel-out/darwin-opt/bin/tensorflow/python/framework/fast_tensor_util.so -Wl,-force_load,bazel-out/darwin-opt/bin/tensorflow/python/_objs/framework/fast_tensor_util.so/fast_tensor_util.o -headerpad_max_install_names -no-canonical-prefixes -undefined dynamic_lookup

Note in particular the -undefined dynamic_lookup seems to have gone missing, and I think that’s causing the link errors.

(Unrelated: the additional mmacosx-min-version is also surprising to me — is there a way to set that minimum version if targeting an older Mac OS than the build host?)

What operating system are you running Bazel on?

Mac OS 10.14.3

What’s the output of bazel info release?

$ bazel info release Starting local Bazel server and connecting to it… release 0.23.0

What’s the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

$ git remote get-url origin ; git rev-parse master ; git rev-parse HEAD https://github.com/tensorflow/tensorflow.git f1bbf1d83e88e044a10066ac7fe1975bf76c9c58 f1bbf1d83e88e044a10066ac7fe1975bf76c9c58

Have you found anything relevant by searching the web?

No.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 28 (15 by maintainers)

Commits related to this issue

Most upvoted comments

Yes, I am waiting for a fix for another release-blocker before creating the next candidate.