tensorflow: Ubuntu 16.04 / Gpu 1080 Version compile Error (Tensorflow installation)

Hello, I recently got a new machine gpu 1080 and I did follow most of instructions via http://textminingonline.com/dive-into-tensorflow-part-iii-gtx-1080-ubuntu16-04-cuda8-0-cudnn5-0-tensorflow. Anyone successfully installed Ubuntu 16.04 GPU with that version?? Let me know

But the Issue is it doesn’t let me complie via bazel.

The following code…

ERROR: /home/ryan/git_ryan/tensorflow/tensorflow/core/kernels/BUILD:1655:1: undeclared inclusion(s) in rule ‘//tensorflow/core/kernels:dense_update_ops_gpu’: this rule is missing dependency declarations for the following files included by ‘tensorflow/core/kernels/dense_update_ops_gpu.cu.cc’: ‘/usr/local/cuda-8.0/include/cuda_runtime.h’ ‘/usr/local/cuda-8.0/include/host_config.h’ ‘/usr/local/cuda-8.0/include/builtin_types.h’ ‘/usr/local/cuda-8.0/include/device_types.h’ ‘/usr/local/cuda-8.0/include/host_defines.h’ ‘/usr/local/cuda-8.0/include/driver_types.h’ ‘/usr/local/cuda-8.0/include/surface_types.h’ ‘/usr/local/cuda-8.0/include/texture_types.h’ ‘/usr/local/cuda-8.0/include/vector_types.h’ ‘/usr/local/cuda-8.0/include/library_types.h’ ‘/usr/local/cuda-8.0/include/channel_descriptor.h’ ‘/usr/local/cuda-8.0/include/cuda_runtime_api.h’ ‘/usr/local/cuda-8.0/include/cuda_device_runtime_api.h’ ‘/usr/local/cuda-8.0/include/driver_functions.h’ ‘/usr/local/cuda-8.0/include/vector_functions.h’ ‘/usr/local/cuda-8.0/include/vector_functions.hpp’ ‘/usr/local/cuda-8.0/include/common_functions.h’ ‘/usr/local/cuda-8.0/include/math_functions.h’ ‘/usr/local/cuda-8.0/include/math_functions.hpp’ ‘/usr/local/cuda-8.0/include/math_functions_dbl_ptx3.h’ ‘/usr/local/cuda-8.0/include/math_functions_dbl_ptx3.hpp’ ‘/usr/local/cuda-8.0/include/cuda_surface_types.h’ ‘/usr/local/cuda-8.0/include/cuda_texture_types.h’ ‘/usr/local/cuda-8.0/include/device_functions.h’ ‘/usr/local/cuda-8.0/include/device_functions.hpp’ ‘/usr/local/cuda-8.0/include/device_atomic_functions.h’ ‘/usr/local/cuda-8.0/include/device_atomic_functions.hpp’ ‘/usr/local/cuda-8.0/include/device_double_functions.h’ ‘/usr/local/cuda-8.0/include/device_double_functions.hpp’ ‘/usr/local/cuda-8.0/include/sm_20_atomic_functions.h’ ‘/usr/local/cuda-8.0/include/sm_20_atomic_functions.hpp’ ‘/usr/local/cuda-8.0/include/sm_32_atomic_functions.h’ ‘/usr/local/cuda-8.0/include/sm_32_atomic_functions.hpp’ ‘/usr/local/cuda-8.0/include/sm_35_atomic_functions.h’ ‘/usr/local/cuda-8.0/include/sm_60_atomic_functions.h’ ‘/usr/local/cuda-8.0/include/sm_60_atomic_functions.hpp’ ‘/usr/local/cuda-8.0/include/sm_20_intrinsics.h’ ‘/usr/local/cuda-8.0/include/sm_20_intrinsics.hpp’ ‘/usr/local/cuda-8.0/include/sm_30_intrinsics.h’ ‘/usr/local/cuda-8.0/include/sm_30_intrinsics.hpp’ ‘/usr/local/cuda-8.0/include/sm_32_intrinsics.h’ ‘/usr/local/cuda-8.0/include/sm_32_intrinsics.hpp’ ‘/usr/local/cuda-8.0/include/sm_35_intrinsics.h’ ‘/usr/local/cuda-8.0/include/surface_functions.h’ ‘/usr/local/cuda-8.0/include/texture_fetch_functions.h’ ‘/usr/local/cuda-8.0/include/texture_indirect_functions.h’ ‘/usr/local/cuda-8.0/include/surface_indirect_functions.h’ ‘/usr/local/cuda-8.0/include/device_launch_parameters.h’ ‘/usr/local/cuda-8.0/include/cuda_fp16.h’ ‘/usr/local/cuda-8.0/include/math_constants.h’ ‘/usr/local/cuda-8.0/include/curand_kernel.h’ ‘/usr/local/cuda-8.0/include/curand.h’ ‘/usr/local/cuda-8.0/include/curand_discrete.h’ ‘/usr/local/cuda-8.0/include/curand_precalc.h’ ‘/usr/local/cuda-8.0/include/curand_mrg32k3a.h’ ‘/usr/local/cuda-8.0/include/curand_mtgp32_kernel.h’ ‘/usr/local/cuda-8.0/include/cuda.h’ ‘/usr/local/cuda-8.0/include/curand_mtgp32.h’ ‘/usr/local/cuda-8.0/include/curand_philox4x32_x.h’ ‘/usr/local/cuda-8.0/include/curand_globals.h’ ‘/usr/local/cuda-8.0/include/curand_uniform.h’ ‘/usr/local/cuda-8.0/include/curand_normal.h’ ‘/usr/local/cuda-8.0/include/curand_normal_static.h’ ‘/usr/local/cuda-8.0/include/curand_lognormal.h’ ‘/usr/local/cuda-8.0/include/curand_poisson.h’ ‘/usr/local/cuda-8.0/include/curand_discrete2.h’. nvcc warning : option ‘–relaxed-constexpr’ has been deprecated and replaced by option ‘–expt-relaxed-constexpr’. nvcc warning : option ‘–relaxed-constexpr’ has been deprecated and replaced by option ‘–expt-relaxed-constexpr’. Target //tensorflow/cc:tutorials_example_trainer failed to build Use --verbose_failures to see the command lines of failed build steps. INFO: Elapsed time: 50.107s, Critical Path: 49.88s

For bugs or installation issues, please provide the following information. The more information you provide, the more easily we will be able to offer help and advice.

Environment info

Operating System: Linux 64bit / GPU 1080

Installed version of CUDA and cuDNN: Cuda 8 / cuDNN: 8.0 (please attach the output of ls -l /path/to/cuda/lib/libcud*):

About this issue

Original URL
State: closed
Created 8 years ago
Comments: 19 (9 by maintainers)

Most upvoted comments

I am not certain I know what pywrap error you are having, but I have Ubuntu 16.04 with CUDA 8.0 / cuDNN 5 running on my system with no problems.

I did manually install the latest NVIDIA 367.35.0 Video Driver (.run) and the (.run) file for CUDA 8.0 as well. Make sure you do not ignore the gcc version error. I did that and everything compiled properly, but python failed to execute a tensor graph. I had to go back and create a symbolic link from gcc-4.9 to gcc for the CUDA build.

A few things to try if you haven’t already as I am guessing a bit since I don’t have the pywrap details:

make sure you do a “bazel clean” ( you might need to use the --expunge_async option as well)
if you are running python from the “tensorflow” repository directory, it is known not to work, change to a different director (not sure why, but it is true)
Search for other CUDA 8.0 issues under tensorflow as there are a few and they may help

If that doesn’t help, I would need more details for your current error.

Mazecreator on Aug 12, 2016

@girving Thank you so much, It finally works. Everything I did was right except I needed to download current version of tensorflow source.

rainmaker712 on Aug 16, 2016

@sungjin712 That “Explicit cast” issue was recently fixed; see https://github.com/tensorflow/tensorflow/issues/3752. Are you at HEAD?

girving on Aug 15, 2016

@sungjin712 look at the CI build folder here: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/ci_build (i do not recommend trying the build without docker which defeats the whole purpose)

The main idea is that we run a standard CI (Continuous Integration) build in a docker image (clean, standard environment) and then copy out the pip wheel file:

install docker and nvidia-docker. have the docker daemon running
make sure you have checkout the intended tensorflow version in your cloned git repo, e.g. git checkout v0.10.0rc0.
go to the ci_build folder and copy this file from my gist: https://gist.github.com/falcondai/7e159cc3cfa1529996f567ca284b00a7#file-docker_build-sh (the original build script removes the container after each build)
make sure the first line in Dockerifle.gpu is the cuda + cudnn version you want: for cuda 8 and cudnn 5, it should be FROM nvidia/cuda:8.0-cudnn5-devel. (Note that this image uses ubuntu 14.04 as base, but i think the wheel file will likely work for ubuntu 16.04. if you really want to build in ubuntu 16.04, you need to look into first building your own cuda image, which should be a matter of changing a line in the cuda image Dockerfile and then building, since all public nvidia/cuda images are built on 14.04: https://hub.docker.com/r/nvidia/cuda/)
$ ./docker_build.sh gpu bash.
now you should be inside a container at /workspace, execute $ ./tensorflow/tools/ci_build/builds/pip.sh gpu (this will take a while). DO NOT close this session!
open another session and look up the container id of your tf_ci.gpu by docker ps
copy the built wheel file from that container, e.g. $ docker cp <container_id>:/workspace/pip_test/whl/tensorflow-0.10.0rc0-cp27-none-linux_x86_64.whl .
pip install this wheel file $ pip install -U <host-path>

falcondai on Aug 14, 2016

You need to update the CROSSTOOLS file to see this CUDA includes: tensorflow/third_party/gpus/crosstool/CROSSTOOL

Around line 65, add: cxx_builtin_include_directory: “/usr/local/cuda-8.0/include”

Mazecreator on Aug 12, 2016