tensorflow: Linker error when compiling from head of master on MacOS

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS Mojave and High Sierra
  • Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
  • TensorFlow installed from (source or binary): source
  • TensorFlow version (use command below): master
  • Python version: 2.7
  • Bazel version (if compiling from source): 0.17…2-homebrew
  • GCC/Compiler version (if compiling from source):
clang -v
Apple LLVM version 10.0.0 (clang-1000.11.45.2)
  • CUDA/cuDNN version: N/A
  • GPU model and memory: N/A
  • Exact command to reproduce:
bazel test --test_output=all --nocache_test_results --config=opt tensorflow/compiler/tests:cpu_tests

(Configured with XLA support enabled)

Describe the problem

When trying to run the above command I get:

ld: can't open -exported_symbols_list file: -filelist
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Source code / logs

After applying this diff:

index df15914233..8de9c7cfdd 100644
--- a/tensorflow/tensorflow.bzl
+++ b/tensorflow/tensorflow.bzl
@@ -1636,8 +1636,7 @@ def tf_py_wrap_cc(
     )
     extra_linkopts = select({
         "@local_config_cuda//cuda:darwin": [
-            "-Wl,-exported_symbols_list",
-            "$(location %s.lds)" % vscriptname,
+            "-Wl,-exported_symbols_list,$(location %s.lds)" % vscriptname,
         ],
         clean_dep("//tensorflow:windows"): [],
         "//conditions:default": [

and running bazel test --test_output=all --nocache_test_results --config=opt tensorflow/compiler/tests:cpu_tests

I get lots of warnings during linking:

ld: warning: cannot export hidden symbol std::__1::shared_ptr<tensorflow::AWSSha256HMACOpenSSLImpl>::__enable_weak_this(...) from bazel-out/darwin-opt/bin/tensorflow/core/platform/s3/libaws_crypto.lo(aws_crypto.o)
ld: warning: cannot export hidden symbol std::__1::shared_ptr<tensorflow::AWSLogSystem>::__enable_weak_this(...) from bazel-out/darwin-opt/bin/tensorflow/core/platform/s3/libaws_logging.lo(aws_logging.o)
ld: warning: cannot export hidden symbol std::__1::shared_ptr<tensorflow::Notification>::__enable_weak_this(...) from bazel-out/darwin-opt/bin/tensorflow/core/grappler/libutils.a(utils.o)

And then when the tests are ran I get errors such as:

2018-10-11 13:02:47.136859: I tensorflow/compiler/xla/service/service.cc:157]   StreamExecutor device (0): <undefined>, <undefined>
*** Received signal 10 ***
*** BEGIN MANGLED STACK TRACE ***
0   libtensorflow_framework.so          0x00000001200870a7 _ZN10tensorflow7testingL17StacktraceHandlerEiP9__siginfoPv + 183
1   libsystem_platform.dylib            0x00007fff66049b3d _sigtramp + 29
2   ???                                 0x0000000000000000 0x0 + 0
3   _pywrap_tensorflow_internal.so      0x00000001143583a9 _ZNSt3__110__function6__funcIZN3xla3cpu13CpuExecutable24ExecuteAsyncOnStreamImplEPKNS2_27ServiceExecutableRunOptionsEN4absl4SpanIKPKNS2_12ShapedBufferEEEPNS2_19HloExecutionProfileEE12AsyncRunTaskNS_9allocatorISH_EEFvvEEclEv + 73
4   libtensorflow_framework.so          0x000000012032bd09 _ZNSt3__110__function6__funcIZN15stream_executor4host10HostStream11EnqueueTaskENS_8functionIFvvEEEE12NotifiedTaskNS_9allocatorIS8_EES6_EclEv + 25
5   libtensorflow_framework.so          0x00000001200641da _ZN5Eigen26NonBlockingThreadPoolTemplIN10tensorflow6thread16EigenEnvironmentEE10WorkerLoopEi + 618
6   libtensorflow_framework.so          0x0000000120063e6f _ZNSt3__110__function6__funcIZN10tensorflow6thread16EigenEnvironment12CreateThreadENS_8functionIFvvEEEEUlvE_NS_9allocatorIS8_EES6_EclEv + 47
7   libtensorflow_framework.so          0x0000000120088a30 _ZNSt3__114__thread_proxyINS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEENS_8functionIFvvEEEEEEEEPvSB_ + 48
8   libsystem_pthread.dylib             0x00007fff6605233d _pthread_body + 126
9   libsystem_pthread.dylib             0x00007fff660552a7 _pthread_start + 70
10  libsystem_pthread.dylib             0x00007fff66051425 thread_start + 13
*** END MANGLED STACK TRACE ***

*** Begin stack trace ***
	tensorflow::CurrentStackTrace()
	tensorflow::testing::StacktraceHandler(int, __siginfo*, void*)
	_sigtramp

	std::__1::__function::__func<xla::cpu::CpuExecutable::ExecuteAsyncOnStreamImpl(xla::ServiceExecutableRunOptions const*, absl::Span<xla::ShapedBuffer const* const>, xla::HloExecutionProfile*)::AsyncRunTask, std::__1::allocator<xla::cpu::CpuExecutable::ExecuteAsyncOnStreamImpl(xla::ServiceExecutableRunOptions const*, absl::Span<xla::ShapedBuffer const* const>, xla::HloExecutionProfile*)::AsyncRunTask>, void ()>::operator()()
	std::__1::__function::__func<stream_executor::host::HostStream::EnqueueTask(std::__1::function<void ()>)::NotifiedTask, std::__1::allocator<stream_executor::host::HostStream::EnqueueTask(std::__1::function<void ()>)::NotifiedTask>, void ()>::operator()()
	Eigen::NonBlockingThreadPoolTempl<tensorflow::thread::EigenEnvironment>::WorkerLoop(int)
	std::__1::__function::__func<tensorflow::thread::EigenEnvironment::CreateThread(std::__1::function<void ()>)::'lambda'(), std::__1::allocator<tensorflow::thread::EigenEnvironment::CreateThread(std::__1::function<void ()>)::'lambda'()>, void ()>::operator()()
	void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, std::__1::function<void ()> > >(void*)
	_pthread_body
	_pthread_start
	thread_start
*** End stack trace ***

Note that cc_wrapper.sh calls gcc however I haven’t changed it so it still points at Apple LLVM.

Note that there is an issue #22759 however that one uses custom code, where as this is HEAD of master.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 24 (10 by maintainers)

Commits related to this issue

Most upvoted comments

Same problem here but not only on master but also r1.12 and r1.11. At the moment I don’t know any version I could compile. Bazel is 0.18.

Still a problem (I’ve updated to the latest Xcode as well - still an issue)

I’d changed string in tensorflow/BUILD instead of tensorflow.bzl. And it works.