tensorflow: tensorflow 2.1 build error

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): source 2.1
TensorFlow version: 2.1.0
Python version: 3.7.6
Installed using virtualenv? pip? conda?: N/A
Bazel version (if compiling from source): 0.29.1
GCC/Compiler version (if compiling from source): 7.4.0
CUDA/cuDNN version: 10.2/ 7.6.5
GPU model and memory: RTX2080Ti GDDR6 11GB

Describe the problem

Provide the exact sequence of commands / steps that you executed before running into the problem

bazel build //tensorflow/tools/pip_package:build_pip_packagee

Any other info / logs Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

ERROR: /home/wmind/repo/tensorflow/tensorflow/python/BUILD:1270:1: C++ compilation of rule '//tensorflow/python:_op_def_registry.so' failed (Exit 1)
In file included from bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/pytypes.h:12:0,
                 from bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/cast.h:13,
                 from bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/attr.h:13,
                 from bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/pybind11.h:44,
                 from ./tensorflow/python/lib/core/pybind11_status.h:21,
                 from tensorflow/python/framework/op_def_registry.cc:20:
bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/detail/common.h:302:12: error: multiple definition of ‘enum class pybind11::return_value_policy’
 enum class return_value_policy : uint8_t {
            ^~~~~~~~~~~~~~~~~~~
In file included from external/pybind11/include/pybind11/pytypes.h:12:0,
                 from external/pybind11/include/pybind11/cast.h:13,
                 from external/pybind11/include/pybind11/attr.h:13,
                 from external/pybind11/include/pybind11/pybind11.h:49,
                 from tensorflow/python/framework/op_def_registry.cc:16:
external/pybind11/include/pybind11/detail/common.h:302:12: note: previous definition here
 enum class return_value_policy : uint8_t {
            ^~~~~~~~~~~~~~~~~~~
In file included from bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/pytypes.h:12:0,
                 from bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/cast.h:13,
                 from bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/attr.h:13,
                 from bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/pybind11.h:44,
                 from ./tensorflow/python/lib/core/pybind11_status.h:21,
                 from tensorflow/python/framework/op_def_registry.cc:20:
bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/detail/common.h: In function ‘constexpr int pybind11::detail::log2(pybind11::size_t, int)’:
bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/detail/common.h:355:29: error: redefinition of ‘constexpr int pybind11::detail::log2(pybind11::size_t, int)’
 inline static constexpr int log2(size_t n, int k = 0) { return (n <= 1) ? k : log2(n >> 1, k + 1); }
                             ^~~~
In file included from external/pybind11/include/pybind11/pytypes.h:12:0,
                 from external/pybind11/include/pybind11/cast.h:13,
                 from external/pybind11/include/pybind11/attr.h:13,
                 from external/pybind11/include/pybind11/pybind11.h:49,
                 from tensorflow/python/framework/op_def_registry.cc:16:
external/pybind11/include/pybind11/detail/common.h:355:29: note: ‘constexpr int pybind11::detail::log2(pybind11::size_t, int)’ previously defined here
 inline static constexpr int log2(size_t n, int k = 0) { return (n <= 1) ? k : log2(n >> 1, k + 1); }
                             ^~~~
In file included from bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/pytypes.h:12:0,
                 from bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/cast.h:13,
                 from bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/attr.h:13,
                 from bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/pybind11.h:44,
                 from ./tensorflow/python/lib/core/pybind11_status.h:21,
                 from tensorflow/python/framework/op_def_registry.cc:20:
bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/detail/common.h: In function ‘constexpr pybind11::size_t pybind11::detail::size_in_ptrs(pybind11::size_t)’:
bazel-out/k8-opt/bin/external/local_config_python/python_include/pybind11/detail/common.h:358:32: error: redefinition of ‘constexpr pybind11::size_t pybind11::detail::size_in_ptrs(pybind11::size_t)’
 inline static constexpr size_t size_in_ptrs(size_t s) { return 1 + ((s - 1) >> log2(sizeof(void *))); }
                                ^~~~~~~~~~~~

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 15 (7 by maintainers)

Most upvoted comments

It’s a bundled libs problem. If pybind11 is installed with pip first, then those headers get picked up by @local_config_python//:python_headers and the pybind headers are duplicated and things go wrong. I’ve unbundled pybind in gentoo and it fixes it, i’ll send a PR upstream soon to fix it in TF too, it needs some reworking because the #include paths are off in TF

perfinion on Jan 27, 2020

I am seeing a very similar error to the one reported here, this time on macOS 10.15 with Xcode 11.3 clang

  exec env - \
    APPLE_SDK_PLATFORM=MacOSX \
    APPLE_SDK_VERSION_OVERRIDE=10.15 \
    PATH=/opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin \
    XCODE_VERSION_OVERRIDE=11.3.0.11C29 \
  external/local_config_cc/wrapped_clang '-D_FORTIFY_SOURCE=1' -fstack-protector -fcolor-diagnostics -Wall -Wthread-safety -Wself-assign -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG '-std=c++11' -iquote . -iquote bazel-out/host/bin -iquote external/com_google_absl -iquote bazel-out/host/bin/external/com_google_absl -iquote external/nsync -iquote bazel-out/host/bin/external/nsync -iquote external/eigen_archive -iquote bazel-out/host/bin/external/eigen_archive -iquote external/local_config_sycl -iquote bazel-out/host/bin/external/local_config_sycl -iquote external/com_googlesource_code_re2 -iquote bazel-out/host/bin/external/com_googlesource_code_re2 -iquote external/gif -iquote bazel-out/host/bin/external/gif -iquote external/libjpeg_turbo -iquote bazel-out/host/bin/external/libjpeg_turbo -iquote external/com_google_protobuf -iquote bazel-out/host/bin/external/com_google_protobuf -iquote external/farmhash_archive -iquote bazel-out/host/bin/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/host/bin/external/fft2d -iquote external/highwayhash -iquote bazel-out/host/bin/external/highwayhash -iquote external/zlib_archive -iquote bazel-out/host/bin/external/zlib_archive -iquote external/local_config_python -iquote bazel-out/host/bin/external/local_config_python -iquote external/pybind11 -iquote bazel-out/host/bin/external/pybind11 -iquote external/mkl_darwin -iquote bazel-out/host/bin/external/mkl_darwin -iquote external/mkl_dnn -iquote bazel-out/host/bin/external/mkl_dnn -iquote external/bazel_tools -iquote bazel-out/host/bin/external/bazel_tools -isystem external/nsync/public -isystem bazel-out/host/bin/external/nsync/public -isystem third_party/eigen3/mkl_include -isystem bazel-out/host/bin/third_party/eigen3/mkl_include -isystem external/eigen_archive -isystem bazel-out/host/bin/external/eigen_archive -isystem external/gif -isystem bazel-out/host/bin/external/gif -isystem external/com_google_protobuf/src -isystem bazel-out/host/bin/external/com_google_protobuf/src -isystem external/farmhash_archive/src -isystem bazel-out/host/bin/external/farmhash_archive/src -isystem external/zlib_archive -isystem bazel-out/host/bin/external/zlib_archive -isystem external/local_config_python/python_include -isystem bazel-out/host/bin/external/local_config_python/python_include -isystem external/pybind11/include -isystem bazel-out/host/bin/external/pybind11/include -isystem external/mkl_darwin/include -isystem bazel-out/host/bin/external/mkl_darwin/include -isystem external/mkl_dnn/include -isystem bazel-out/host/bin/external/mkl_dnn/include -isystem external/mkl_dnn/src -isystem bazel-out/host/bin/external/mkl_dnn/src -isystem external/mkl_dnn/src/common -isystem bazel-out/host/bin/external/mkl_dnn/src/common -isystem external/mkl_dnn/src/cpu -isystem bazel-out/host/bin/external/mkl_dnn/src/cpu -isystem external/mkl_dnn/src/cpu/gemm -isystem bazel-out/host/bin/external/mkl_dnn/src/cpu/gemm -isystem external/mkl_dnn/src/cpu/xbyak -isystem bazel-out/host/bin/external/mkl_dnn/src/cpu/xbyak -MD -MF bazel-out/host/bin/tensorflow/python/_objs/_pywrap_transform_graph.so/transform_graph_wrapper.d -D__CLANG_SUPPORT_DYN_ANNOTATION__ -DEIGEN_MPL2_ONLY '-DEIGEN_MAX_ALIGN_BYTES=64' '-DEIGEN_HAS_TYPE_TRAITS=0' '-frandom-seed=bazel-out/host/bin/tensorflow/python/_objs/_pywrap_transform_graph.so/transform_graph_wrapper.o' -isysroot __BAZEL_XCODE_SDKROOT__ '-mmacosx-version-min=10.15' -g0 '-march=x86-64' -g0 '-std=c++14' -fexceptions '-fvisibility=hidden' -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c tensorflow/python/util/transform_graph_wrapper.cc -o bazel-out/host/bin/tensorflow/python/_objs/_pywrap_transform_graph.so/transform_graph_wrapper.o)
Execution platform: @bazel_tools//platforms:host_platform
In file included from tensorflow/python/util/transform_graph_wrapper.cc:22:
In file included from ./tensorflow/python/lib/core/pybind11_status.h:21:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/pybind11.h:44:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/attr.h:13:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/cast.h:13:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/pytypes.h:12:
bazel-out/host/bin/external/local_config_python/python_include/pybind11/detail/common.h:302:12: error: redefinition of 'return_value_policy'
enum class return_value_policy : uint8_t {
           ^
external/pybind11/include/pybind11/detail/common.h:302:12: note: previous definition is here
enum class return_value_policy : uint8_t {
           ^
In file included from tensorflow/python/util/transform_graph_wrapper.cc:22:
In file included from ./tensorflow/python/lib/core/pybind11_status.h:21:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/pybind11.h:44:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/attr.h:13:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/cast.h:13:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/pytypes.h:12:
bazel-out/host/bin/external/local_config_python/python_include/pybind11/detail/common.h:355:48: error: redefinition of default argument
inline static constexpr int log2(size_t n, int k = 0) { return (n <= 1) ? k : log2(n >> 1, k + 1); }
                                               ^   ~
external/pybind11/include/pybind11/detail/common.h:355:48: note: previous definition is here
inline static constexpr int log2(size_t n, int k = 0) { return (n <= 1) ? k : log2(n >> 1, k + 1); }
                                               ^   ~
In file included from tensorflow/python/util/transform_graph_wrapper.cc:22:
In file included from ./tensorflow/python/lib/core/pybind11_status.h:21:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/pybind11.h:44:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/attr.h:13:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/cast.h:13:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/pytypes.h:12:
bazel-out/host/bin/external/local_config_python/python_include/pybind11/detail/common.h:358:32: error: redefinition of 'size_in_ptrs'
inline static constexpr size_t size_in_ptrs(size_t s) { return 1 + ((s - 1) >> log2(sizeof(void *))); }
                               ^
external/pybind11/include/pybind11/detail/common.h:358:32: note: previous definition is here
inline static constexpr size_t size_in_ptrs(size_t s) { return 1 + ((s - 1) >> log2(sizeof(void *))); }
                               ^
In file included from tensorflow/python/util/transform_graph_wrapper.cc:22:
In file included from ./tensorflow/python/lib/core/pybind11_status.h:21:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/pybind11.h:44:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/attr.h:13:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/cast.h:13:
In file included from bazel-out/host/bin/external/local_config_python/python_include/pybind11/pytypes.h:12:
bazel-out/host/bin/external/local_config_python/python_include/pybind11/detail/common.h:366:18: error: redefinition of 'instance_simple_holder_in_ptrs'
constexpr size_t instance_simple_holder_in_ptrs() {
                 ^
external/pybind11/include/pybind11/detail/common.h:366:18: note: previous definition is here
constexpr size_t instance_simple_holder_in_ptrs() {
                 ^
<snip>

issue appears to be due to two different locations for pybind11 being used.

cjones051073 on Jan 10, 2020

@mihaimaruseac

build succeeded without tensorrt in master

alanpurple on Jan 22, 2020

I can’t in the near future, I’m currently compiling v2.1.0 patched to expect pybind11 v2.4.3 to see if it completes when the versions match.

What I think is happening here is that tensorflow/workspace.bzl specifies pybind11 v2.3.0 but some other part of the machinery just resolves a pybind11 dependency as latest. This would have stopped working reliably on Sep 20, 2019 when v2.4.0 became the latest.

As of this writing master still specifies pybind11 v2.3.0 so for it to work here such a dependency would have to have been fixed or removed.

jok-ts on Jan 22, 2020