tensorflow: bazel still report error: name 'http_archive' is not defined

I’m experiencing the same error described in issue #27189, when I try to build TF v1.12.2 using Bazel 0.25.2 on a Jetson TX2

$ export TF_NEED_CUDA=1
$ export TF_CUDA_VERSION=9.0
$ export CUDA_TOOLKIT_PATH=/usr/local/cuda
$ export TF_CUDNN_VERSION=7.1.5
$ export CUDNN_INSTALL_PATH=/usr/lib/aarch64-linux-gnu/
$ export TF_CUDA_COMPUTE_CAPABILITIES=6.2

$ bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

WARNING: The following configs were expanded more than once: [cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.
ERROR: /home/nvidia/git/tensorflow/WORKSPACE:3:1: name 'http_archive' is not defined
ERROR: Error evaluating WORKSPACE file
ERROR: error loading package '': Encountered error while reading extension file 'closure/defs.bzl': no such package '@io_bazel_rules_closure//closure': error loading package 'external': Could not load //external package
ERROR: error loading package '': Encountered error while reading extension file 'closure/defs.bzl': no such package '@io_bazel_rules_closure//closure': error loading package 'external': Could not load //external package
INFO: Elapsed time: 0.222s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 20

Most upvoted comments

@mihaimaruseac In that case, then I’m going to go with the TF-1.13.1 precompiled binary version from Nvidia and save myself from the trouble of spending 6+ hours building it from source. Thank you for time and assistance with this.

With -mfpu=neon removed, the build ran longer, BUT after almost 6 hours running, then it failed with some kind of native TensorFlow runtime error:

ERROR: /home/nvidia/git/tensorflow/tensorflow/BUILD:533:1: Executing genrule //tensorflow:tf_python_api_gen_v1 failed (Exit 1): bash failed: error executing command 
  (cd /home/nvidia/.cache/bazel/_bazel_nvidia/01c445c7b00bca0241913a79fcd99718/execroot/org_tensorflow && \
  exec env - \
    LD_LIBRARY_PATH=/home/nvidia/ros_ws/devel/lib:/opt/ros/kinetic/lib:/opt/ros/kinetic/lib/aarch64-linux-gnu \
    PATH=/opt/ros/kinetic/bin:/usr/local/cuda-9.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; bazel-out/host/bin/tensorflow/create_tensorflow.python_api_1 --root_init_template=tensorflow/api_template.__init__.py --apidir=bazel-out/host/genfiles/tensorflow_api/v1/ --apiname=tensorflow --apiversion=1 --package=tensorflow.python --output_package=tensorflow._api.v1 bazel-out/host/genfiles/tensorflow/_api/v1/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/app/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/bitwise/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/compat/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/data/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/data/experimental/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/debugging/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/distributions/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/dtypes/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/errors/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/feature_column/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/gfile/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/graph_util/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/image/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/io/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/initializers/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/activations/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/applications/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/applications/densenet/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/applications/inception_resnet_v2/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/applications/inception_v3/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/applications/mobilenet/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/applications/mobilenet_v2/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/applications/nasnet/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/applications/resnet50/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/applications/vgg16/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/applications/vgg19/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/applications/xception/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/backend/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/callbacks/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/constraints/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/datasets/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/datasets/boston_housing/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/datasets/cifar10/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/datasets/cifar100/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/datasets/fashion_mnist/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/datasets/imdb/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/datasets/mnist/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/datasets/reuters/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/estimator/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/initializers/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/layers/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/losses/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/metrics/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/models/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/optimizers/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/preprocessing/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/preprocessing/image/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/preprocessing/sequence/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/preprocessing/text/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/regularizers/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/utils/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/wrappers/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/keras/wrappers/scikit_learn/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/layers/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/linalg/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/logging/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/losses/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/manip/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/math/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/metrics/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/nn/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/nn/rnn_cell/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/profiler/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/python_io/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/quantization/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/random/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/resource_loader/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/strings/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/saved_model/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/saved_model/builder/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/saved_model/constants/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/saved_model/loader/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/saved_model/main_op/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/saved_model/signature_constants/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/saved_model/signature_def_utils/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/saved_model/tag_constants/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/saved_model/utils/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/sets/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/sparse/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/spectral/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/summary/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/sysconfig/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/test/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/train/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/train/queue_runner/__init__.py bazel-out/host/genfiles/tensorflow/_api/v1/user_ops/__init__.py')
Traceback (most recent call last):
  File "/home/nvidia/.cache/bazel/_bazel_nvidia/01c445c7b00bca0241913a79fcd99718/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api_1.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/home/nvidia/.cache/bazel/_bazel_nvidia/01c445c7b00bca0241913a79fcd99718/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api_1.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/home/nvidia/.cache/bazel/_bazel_nvidia/01c445c7b00bca0241913a79fcd99718/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api_1.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/usr/lib/python3.5/imp.py", line 242, in load_module
    return load_dynamic(name, filename, file)
  File "/usr/lib/python3.5/imp.py", line 342, in load_dynamic
    return _load(spec)
ImportError: /home/nvidia/.cache/bazel/_bazel_nvidia/01c445c7b00bca0241913a79fcd99718/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api_1.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so: undefined symbol: _ZN3Aws11Environment6GetEnvB5cxx11EPKc

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nvidia/.cache/bazel/_bazel_nvidia/01c445c7b00bca0241913a79fcd99718/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api_1.runfiles/org_tensorflow/tensorflow/python/tools/api/generator/create_python_api.py", line 27, in <module>
    from tensorflow.python.tools.api.generator import doc_srcs
  File "/home/nvidia/.cache/bazel/_bazel_nvidia/01c445c7b00bca0241913a79fcd99718/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api_1.runfiles/org_tensorflow/tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/home/nvidia/.cache/bazel/_bazel_nvidia/01c445c7b00bca0241913a79fcd99718/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api_1.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):
  File "/home/nvidia/.cache/bazel/_bazel_nvidia/01c445c7b00bca0241913a79fcd99718/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api_1.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/home/nvidia/.cache/bazel/_bazel_nvidia/01c445c7b00bca0241913a79fcd99718/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api_1.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/home/nvidia/.cache/bazel/_bazel_nvidia/01c445c7b00bca0241913a79fcd99718/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api_1.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/usr/lib/python3.5/imp.py", line 242, in load_module
    return load_dynamic(name, filename, file)
  File "/usr/lib/python3.5/imp.py", line 342, in load_dynamic
    return _load(spec)
ImportError: /home/nvidia/.cache/bazel/_bazel_nvidia/01c445c7b00bca0241913a79fcd99718/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api_1.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so: undefined symbol: _ZN3Aws11Environment6GetEnvB5cxx11EPKc


Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/errors

for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 25367.470s, Critical Path: 430.44s
INFO: 8521 processes: 8521 local.
FAILED: Build did NOT complete successfully

Ok, so for AArch64 -mfpu=neon is not needed as neon is mandatory and included by default. Still dubious that the compiler would complain instead of silently ignoring but not the topic to discuss that here.

I think it is better to run ./configure again as you changed some options. Though these seem to be in Bazel files, so probably there won’t be any change. Better safe though, so I recommend rerunning.

I got builds taking 4-5 hours at one point, it all depends on how many CPUs and how powerful they are. The codebase is quite big.

You can build without NCCL. In fact, many times I built without any GPU at all.