tensorflow: Linker error when compiling from head of master on MacOS
System information
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS Mojave and High Sierra
- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
- TensorFlow installed from (source or binary): source
- TensorFlow version (use command below): master
- Python version: 2.7
- Bazel version (if compiling from source): 0.17…2-homebrew
- GCC/Compiler version (if compiling from source):
clang -v
Apple LLVM version 10.0.0 (clang-1000.11.45.2)
- CUDA/cuDNN version: N/A
- GPU model and memory: N/A
- Exact command to reproduce:
bazel test --test_output=all --nocache_test_results --config=opt tensorflow/compiler/tests:cpu_tests
(Configured with XLA support enabled)
Describe the problem
When trying to run the above command I get:
ld: can't open -exported_symbols_list file: -filelist
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Source code / logs
After applying this diff:
index df15914233..8de9c7cfdd 100644
--- a/tensorflow/tensorflow.bzl
+++ b/tensorflow/tensorflow.bzl
@@ -1636,8 +1636,7 @@ def tf_py_wrap_cc(
)
extra_linkopts = select({
"@local_config_cuda//cuda:darwin": [
- "-Wl,-exported_symbols_list",
- "$(location %s.lds)" % vscriptname,
+ "-Wl,-exported_symbols_list,$(location %s.lds)" % vscriptname,
],
clean_dep("//tensorflow:windows"): [],
"//conditions:default": [
and running bazel test --test_output=all --nocache_test_results --config=opt tensorflow/compiler/tests:cpu_tests
I get lots of warnings during linking:
ld: warning: cannot export hidden symbol std::__1::shared_ptr<tensorflow::AWSSha256HMACOpenSSLImpl>::__enable_weak_this(...) from bazel-out/darwin-opt/bin/tensorflow/core/platform/s3/libaws_crypto.lo(aws_crypto.o)
ld: warning: cannot export hidden symbol std::__1::shared_ptr<tensorflow::AWSLogSystem>::__enable_weak_this(...) from bazel-out/darwin-opt/bin/tensorflow/core/platform/s3/libaws_logging.lo(aws_logging.o)
ld: warning: cannot export hidden symbol std::__1::shared_ptr<tensorflow::Notification>::__enable_weak_this(...) from bazel-out/darwin-opt/bin/tensorflow/core/grappler/libutils.a(utils.o)
And then when the tests are ran I get errors such as:
2018-10-11 13:02:47.136859: I tensorflow/compiler/xla/service/service.cc:157] StreamExecutor device (0): <undefined>, <undefined>
*** Received signal 10 ***
*** BEGIN MANGLED STACK TRACE ***
0 libtensorflow_framework.so 0x00000001200870a7 _ZN10tensorflow7testingL17StacktraceHandlerEiP9__siginfoPv + 183
1 libsystem_platform.dylib 0x00007fff66049b3d _sigtramp + 29
2 ??? 0x0000000000000000 0x0 + 0
3 _pywrap_tensorflow_internal.so 0x00000001143583a9 _ZNSt3__110__function6__funcIZN3xla3cpu13CpuExecutable24ExecuteAsyncOnStreamImplEPKNS2_27ServiceExecutableRunOptionsEN4absl4SpanIKPKNS2_12ShapedBufferEEEPNS2_19HloExecutionProfileEE12AsyncRunTaskNS_9allocatorISH_EEFvvEEclEv + 73
4 libtensorflow_framework.so 0x000000012032bd09 _ZNSt3__110__function6__funcIZN15stream_executor4host10HostStream11EnqueueTaskENS_8functionIFvvEEEE12NotifiedTaskNS_9allocatorIS8_EES6_EclEv + 25
5 libtensorflow_framework.so 0x00000001200641da _ZN5Eigen26NonBlockingThreadPoolTemplIN10tensorflow6thread16EigenEnvironmentEE10WorkerLoopEi + 618
6 libtensorflow_framework.so 0x0000000120063e6f _ZNSt3__110__function6__funcIZN10tensorflow6thread16EigenEnvironment12CreateThreadENS_8functionIFvvEEEEUlvE_NS_9allocatorIS8_EES6_EclEv + 47
7 libtensorflow_framework.so 0x0000000120088a30 _ZNSt3__114__thread_proxyINS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEENS_8functionIFvvEEEEEEEEPvSB_ + 48
8 libsystem_pthread.dylib 0x00007fff6605233d _pthread_body + 126
9 libsystem_pthread.dylib 0x00007fff660552a7 _pthread_start + 70
10 libsystem_pthread.dylib 0x00007fff66051425 thread_start + 13
*** END MANGLED STACK TRACE ***
*** Begin stack trace ***
tensorflow::CurrentStackTrace()
tensorflow::testing::StacktraceHandler(int, __siginfo*, void*)
_sigtramp
std::__1::__function::__func<xla::cpu::CpuExecutable::ExecuteAsyncOnStreamImpl(xla::ServiceExecutableRunOptions const*, absl::Span<xla::ShapedBuffer const* const>, xla::HloExecutionProfile*)::AsyncRunTask, std::__1::allocator<xla::cpu::CpuExecutable::ExecuteAsyncOnStreamImpl(xla::ServiceExecutableRunOptions const*, absl::Span<xla::ShapedBuffer const* const>, xla::HloExecutionProfile*)::AsyncRunTask>, void ()>::operator()()
std::__1::__function::__func<stream_executor::host::HostStream::EnqueueTask(std::__1::function<void ()>)::NotifiedTask, std::__1::allocator<stream_executor::host::HostStream::EnqueueTask(std::__1::function<void ()>)::NotifiedTask>, void ()>::operator()()
Eigen::NonBlockingThreadPoolTempl<tensorflow::thread::EigenEnvironment>::WorkerLoop(int)
std::__1::__function::__func<tensorflow::thread::EigenEnvironment::CreateThread(std::__1::function<void ()>)::'lambda'(), std::__1::allocator<tensorflow::thread::EigenEnvironment::CreateThread(std::__1::function<void ()>)::'lambda'()>, void ()>::operator()()
void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, std::__1::function<void ()> > >(void*)
_pthread_body
_pthread_start
thread_start
*** End stack trace ***
Note that cc_wrapper.sh
calls gcc
however I haven’t changed it so it still points at Apple LLVM.
Note that there is an issue #22759 however that one uses custom code, where as this is HEAD of master.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 24 (10 by maintainers)
Commits related to this issue
- Updated the nGraph version and fixed the broken build for MacOS as reported in issue #22902 — committed to NervanaSystems/tensorflow by avijit-nervana 6 years ago
- Fix linker parameter syntax when passing -exported_symbols_list See https://github.com/tensorflow/tensorflow/issues/22902 — committed to reuben/tensorflow by reuben 5 years ago
- Fix linker parameter syntax when passing -exported_symbols_list See https://github.com/tensorflow/tensorflow/issues/22902 — committed to reuben/tensorflow by reuben 5 years ago
- Fix linker parameter syntax when passing -exported_symbols_list See https://github.com/tensorflow/tensorflow/issues/22902 — committed to reuben/tensorflow by reuben 5 years ago
- Fix linker parameter syntax when passing -exported_symbols_list See https://github.com/tensorflow/tensorflow/issues/22902 — committed to reuben/tensorflow by reuben 5 years ago
Same problem here but not only on master but also r1.12 and r1.11. At the moment I don’t know any version I could compile. Bazel is 0.18.
Still a problem (I’ve updated to the latest Xcode as well - still an issue)
I’d changed string in tensorflow/BUILD instead of tensorflow.bzl. And it works.