tensorflow: Custom op kernels compiled with C++17 are binary incompatible with tensorflow
- Have I written custom code: yes
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux CentOS 7.5 (in docker)
- TensorFlow installed from (source or binary): pip binary
- TensorFlow version (use command below): 1.11
- Python version: 3.6.4
- GCC/Compiler version (if compiling from source): 7.3.1
current behavior When compiling a custom op kernel with c++17 features, compilation succeeds, but loading the generated library fails at runtime with the following error:
tensorflow.python.framework.errors_impl.NotFoundError: /usr/local/lib64/python3.6/site-packages/package-0.0.1-py3.6-linux-x86_64.egg/_package_ops.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN10tensorflow11GetNodeAttrERKNS_9AttrSliceESt17basic_string_viewIcSt11char_traitsIcEEPSs
expected behavior Loading the library should succeed
Code to reproduce the issue
Any custom op using GetAttr should reproduce this, e.g. the custom op in the custom op tutorial
Other info / logs
This symbol that is undefined is tensorflow::GetNodeAttr(tensorflow::AttrSlice const&, std::basic_string_view<char, std::char_traits<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >*)
the symbol that it should have linked against (which exists in libtensorflow_framework.so) is tensorflow::GetNodeAttr(tensorflow::AttrSlice const&, absl::string_view, std::basic_string<char, std::char_traits<char>, std::allocator<char> >*)
the difference is std::basic_string<char, std::char_traits<char> vs. absl::string_view for the second argument.
It appears to be this way because https://github.com/abseil/abseil-cpp/blob/master/absl/strings/string_view.h uses std::string_view to provide absl::string_view when std::string_view is available (i.e. when compiling using c++17 standard)
The tensorflow binary is compiled with c++11, so it has absl::string_view provided by the custom absl implementation, whereas my custom op, compiled with c++17 gets the standard library implementation, so they are binary incompatible.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 24 (16 by maintainers)
Commits related to this issue
- Remove call to std::random_shuffle. std::random_shuffle is removed in C++17, and is causing TF to be not buildable in such configurations. Helps with Issue #23561 PiperOrigin-RevId: 231984187 — committed to tensorflow/tensorflow by gunan 5 years ago
- C++17 build without linking libc++ Based on #23561 and #41710, trying to see if this would enable building on C++17 without also linking in `libc++` (which is a Clang lib, does not come from a defaul... — committed to tensorflow/tensorflow by mihaimaruseac 4 years ago
I think the
-config=c++1z/17will add-stdlib=libc++to the params, which is only recognized byClang, is there some solution tocustom_opissue when building with GCC 7 and C++17 in Ubuntu 16.04?I had some spare time to try out, but I ran into build issues when trying to rebuild tf with gcc 7. So had to start going down the rabbit hole and got distracted. I will retry on a macos.
Thank you very much for sharing this! I have not been able to look at this yet, but this is a priority for me. Also cc @yifeif
a workaround is to do
before