tensorflow: tensorflow/core/framework/op_def.pb.h:10:40: fatal error: google/protobuf/port_def.inc: No such file or directory

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): use plugin
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): tensorflow/tensorflow:latest-gpu-py3
TensorFlow version (use command below): 1.14.1-dev20190409
Python version: 3.5
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version:
GPU model and memory:

You can collect some of this information using our environment capture script You can also obtain the TensorFlow version with python -c “import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)”

Describe the current behavior As of 1.14.1-dev20190409, custom plugins fail to build with the following error:

  In file included from /usr/local/lib/python2.7/dist-packages/tensorflow/include/tensorflow/core/framework/op_def_builder.h:24:0,
                   from /usr/local/lib/python2.7/dist-packages/tensorflow/include/tensorflow/core/framework/op.h:23,
                   from horovod/tensorflow/mpi_ops.cc:22:
  /usr/local/lib/python2.7/dist-packages/tensorflow/include/tensorflow/core/framework/op_def.pb.h:10:40: fatal error: google/protobuf/port_def.inc: No such file or directory
   #include <google/protobuf/port_def.inc>
                                          ^
  compilation terminated.

This seems to be related to https://github.com/tensorflow/tensorflow/commit/6168f476b52d6d40eeff1823943ed2c0ea28adde. It appears that not all the files are placed in the proper locations after the installation.

Describe the expected behavior Plugins should build successfully.

Code to reproduce the issue Provide a reproducible test case that is the bare minimum necessary to generate the problem.

$ docker run -it --rm tensorflow/tensorflow:latest-gpu-py3
# apt install -y mpich
# HOROVOD_WITH_TENSORFLOW=1 pip install -v horovod

Other info / logs Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

cc @martinwicke @gunan

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 2
Comments: 47 (34 by maintainers)

Commits related to this issue

Add missing header file for protobuf 1. For third_party/systemlibs/protobuf.BUILD 2. When creating the pip package, we should also copy "*.inc" files for protobuf. Fixes #27697 PiperOrigin-RevId: 2... — committed to Cache-Cloud/tensorflow by tensorflower-gardener 5 years ago
Fix setup.py to also include *.inc files for protobuf Fixes #27697 PiperOrigin-RevId: 243634475 — committed to Cache-Cloud/tensorflow by tensorflower-gardener 5 years ago
Fix `find protobuf_archive` command for macos The original command doesn't work for macos because regex behaves differently. https://stackoverflow.com/questions/5905493/why-isnt-this-regex-working-fi... — committed to erwa/tensorflow by tensorflower-gardener 5 years ago
Fix find protobuf_archive Removing -type flag because it is unnecessary and on some platform it doesn't accepted multiple types, which caused: find: Arguments to -type should contain only one letter ... — committed to testkevinbonz/tensorflow by tensorflower-gardener 5 years ago

Most upvoted comments

Looks like the problem is when creating the pip package, we missed to copy header files that has .inc extension. https://github.com/tensorflow/tensorflow/blob/1cdae4922cfe6821eafb163d9f825e29b4f1fc80/tensorflow/tools/pip_package/build_pip_package.sh#L128-L130

I’ll send a fix for this.

meteorcloudy on Apr 11, 2019

Hi, I experience this problem with 1.14.0 . From what I see in the sources, as well as bananabowl’s comment / reference to the cherry picks, the needed changes are in 1.14.0. Still getting the problem. Here is the error:

In file included from ./deps/tensorflow-headers-1.14.0/tensorflow/core/framework/tensor_shape.h:22:0,
                 from ./deps/tensorflow-headers-1.14.0/tensorflow/core/framework/tensor.h:23,
                 from ./deps/tensorflow-headers-1.14.0/tensorflow/cc/framework/ops.h:21,
                 from ./deps/tensorflow-headers-1.14.0/tensorflow/cc/client/client_session.h:24,
                 from ./include/detect_tensorflow.hpp:5,
                 from src/detect_tensorflow.cpp:1:
./deps/tensorflow-headers-1.14.0/tensorflow/core/framework/types.pb.h:10:10: fatal error: google/protobuf/port_def.inc: No such file or directory
 #include <google/protobuf/port_def.inc>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I am using the headers generated by //tensorflow:install_headers . Platform : AARCH64 OS : Ubuntu 18.04 LTS Hardware: Nvidia Xavier AGX Software: TensorFlow 1.14.0 from a source release. Compiled with Bazel 0.24.1 (from a dist package).

andreyhristov on Jun 24, 2019

@meteorcloudy, I can confirm that both Mac and Linux nightly builds are operational. Could you pick this fix to the 1.4 branch?

alsrgv on Apr 28, 2019

@alsrgv Thanks! Hopefully this time it will work

meteorcloudy on Apr 26, 2019

@seanpmorgan Looks like regex in find does behave differently on macos: https://stackoverflow.com/questions/5905493/why-isnt-this-regex-working-find-regex-m-h

https://github.com/tensorflow/tensorflow/blob/d65760b805cdd729743ba3d7c7b4ae8897304dd6/tensorflow/tools/pip_package/build_pip_package.sh#L128

I will change it to

find protobuf_archive -type f,l \( -name "*.h" -o -name "*.inc" \)

meteorcloudy on Apr 24, 2019