tensorflow: Docker build for TensorFlow GPU taking too long(Up 45 hours!)
Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template
System information
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04.3 LTS
- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
- TensorFlow installed from (source or binary): Docker image (tensorflow/tensorflow:devel-gpu-py3)
- TensorFlow version: 2.0
- Python version: 3
- Installed using virtualenv? pip? conda?: Docker
- Bazel version (if compiling from source): default for docker image
- GCC/Compiler version (if compiling from source): default for docker image
- CUDA/cuDNN version: 10.0
- GPU model and memory: GT 750m
- CPU: Intel® Core™ i7-4500U CPU @ 1.80GHz × 4
Describe the problem I’m trying to build TensorFlow for my GPU with CUDA compute capability 3.0. The build seems to take forever to complete. I’m following these guides: Build from source using Docker Setup Docker for TensorFlow
I can’t figure out if I did something wrong.
Provide the exact sequence of commands/steps that you executed before running into the problem
Following Docker commands were used to run the container:
docker pull tensorflow/tensorflow:devel-gpu-py3
docker run --gpus all -it -w /tensorflow -v $PWD:/mnt -e HOST_PERMS="$(id -u):$(id -g)" tensorflow/tensorflow:devel-gpu-py3 bash
Then navigating to tensorflow_src, ./configure the command was run setting all values to defaults and “No” to all features(CUDA and CUdnn were automatically detected)
Finally, the following command was run to start the build process:
bazel build --config=v2 --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
Any other info / logs Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
Currently, the container has been up for 45 hours!. It doesn’t seem like it is coming to end anytime soon. At the time of writing Bazel build is on action 29,910/29,333.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9bcd8e96c6ce tensorflow/tensorflow:devel-gpu-py3 "bash" 45 hours ago Up 45 hours cool_spence
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 19 (6 by maintainers)
@RafayAK : I would just use “tensorflow/tensorflow:devel-gpu” and add python3 + (required python packages) as this is a lot easier then gathering the cuda requirements. Additionally I would open a new issue about the missing updates of the “tensorflow/tensorflow:devel-gpu-py3” image.
use the update and the upgrade and then run the code in the docker cloud.