tensorflow: tensorflow 1.14 build failed

System information

OS Platform and Distribution (Linux Ubuntu 18.04):
TensorFlow installed from (source):
TensorFlow version: r1.14
Python version: v3.6.8
Bazel version (tried v0.24.1, v0.26, v0.28):
GCC/Compiler version (gcc version 7.4):
CUDA/cuDNN version: 10.0/7.5
GPU model and memory: GTX1050

Describe the problem I tried to use below command to build concat_op_test application. bazel test -c opt --strip=never -s //tensorflow/core/kernels:concat_op_test --verbose_failures The compiler I select is clang and build with CUDA support. I got below errors when bazel version is 0.24.1 and 0.26

ERROR: Skipping '//tensorflow/core/kernels:concat_op_test': error loading package 'tensorflow/core/kernels': Encountered error while reading extension file 'cu
da/build_defs.bzl': no such package '@local_config_cuda//cuda': Traceback (most recent call last):
        File "/home/chengleiwang/code/tensorflow/third_party/gpus/cuda_configure.bzl", line 1266
                _create_local_cuda_repository(repository_ctx)
        File "/home/chengleiwang/code/tensorflow/third_party/gpus/cuda_configure.bzl", line 1199, in _create_local_cuda_repository
                _tpl(repository_ctx, "crosstool:BUILD", c...)
        File "/home/chengleiwang/code/tensorflow/third_party/gpus/cuda_configure.bzl", line 745, in _tpl
                repository_ctx.template(out, Label(("//third_party/gpus/%s...)), ...)
class com.google.devtools.build.lib.syntax.SkylarkList$MutableList cannot be cast to class java.lang.String (com.google.devtools.build.lib.syntax.SkylarkList$MutableList is in unnamed module of loader 'app'; java.lang.String is in module java.base of loader 'bootstrap')
ERROR: error loading package 'tensorflow/core/kernels': Encountered error while reading extension file 'cuda/build_defs.bzl': no such package '@local_config_cu
da//cuda': Traceback (most recent call last):
        File "/home/chengleiwang/code/tensorflow/third_party/gpus/cuda_configure.bzl", line 1266
                _create_local_cuda_repository(repository_ctx)
        File "/home/chengleiwang/code/tensorflow/third_party/gpus/cuda_configure.bzl", line 1199, in _create_local_cuda_repository
                _tpl(repository_ctx, "crosstool:BUILD", c...)
        File "/home/chengleiwang/code/tensorflow/third_party/gpus/cuda_configure.bzl", line 745, in _tpl
                repository_ctx.template(out, Label(("//third_party/gpus/%s...)), ...)
class com.google.devtools.build.lib.syntax.SkylarkList$MutableList cannot be cast to class java.lang.String (com.google.devtools.build.lib.syntax.SkylarkList$MutableList is in unnamed module of loader 'app'; java.lang.String is in module java.base of loader 'bootstrap')

I got below errors when bazel version is 0.28.0 (I changed the bazel version check in configure.py current_bazel_version = check_bazel_version('0.24.1', '0.28.2') )

Starting local Bazel server and connecting to it...
Internal error thrown during build. Printing stack trace: java.lang.RuntimeException: Unrecoverable error while evaluating node 'REPOSITORY_DIREC
TORY:@local_config_cuda' (requested by nodes 'REPOSITORY:@local_config_cuda')
        at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:528)
        at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:399)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ClassCastException: class com.google.devtools.build.lib.syntax.SkylarkList$MutableList cannot be cast to class java.lang.String (com.google.devtools.build.lib.syntax.SkylarkList$MutableList is in unnamed module of loader 'app'; java.lang.String is in module java.base of loader 'bootstrap')
        at com.google.devtools.build.lib.bazel.repository.skylark.SkylarkRepositoryContext.createFileFromTemplate(SkylarkRepositoryContext.java:295)
        at jdk.internal.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.base/java.lang.reflect.Method.invoke(Unknown Source)
        at com.google.devtools.build.lib.syntax.MethodDescriptor.call(MethodDescriptor.java:135)
        at com.google.devtools.build.lib.syntax.FuncallExpression.invokeObjectMethod(FuncallExpression.java:930)
        at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:898)
        at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
        at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:201)
        at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
        at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
        at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:475)
        at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:437)
        at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
        at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
        at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
        at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:201)
        at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
        at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
        at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:475)
        at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:437)
        at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
        at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
        at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
        at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:201)
        at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
        at com.google.devtools.build.lib.syntax.Eval.execStatements(Eval.java:231)
        at com.google.devtools.build.lib.syntax.Eval.execIf(Eval.java:144)
        at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:214)
        at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
        at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
        at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:475)
        at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:437)
        at com.google.devtools.build.lib.bazel.repository.skylark.SkylarkRepositoryFunction.fetch(SkylarkRepositoryFunction.java:167)
        at com.google.devtools.build.lib.rules.repository.RepositoryDelegatorFunction.fetchRepository(RepositoryDelegatorFunction.java:288)
        at com.google.devtools.build.lib.rules.repository.RepositoryDelegatorFunction.compute(RepositoryDelegatorFunction.java:215)
        at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:451)
        ... 4 more

INFO: Elapsed time: 2.858s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
    currently loading: tensorflow/core/kernels
    Fetching @local_config_cuda; fetching
Internal error thrown during build. Printing stack trace: java.lang.RuntimeException: Unrecoverable error while evaluating node 'REPOSITORY_DIRECTORY:@local_config_cuda' (requested by nodes 'REPOSITORY:@local_config_cuda')
        at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:528)
        at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:399)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ClassCastException: class com.google.devtools.build.lib.syntax.SkylarkList$MutableList cannot be cast to class java.lang.String (com.google.devtools.build.lib.syntax.SkylarkList$MutableList is in unnamed module of loader 'app'; java.lang.String is in module java.base of loader 'bootstrap')
        at com.google.devtools.build.lib.bazel.repository.skylark.SkylarkRepositoryContext.createFileFromTemplate(SkylarkRepositoryContext.java:295)
        at jdk.internal.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.base/java.lang.reflect.Method.invoke(Unknown Source)
        at com.google.devtools.build.lib.syntax.MethodDescriptor.call(MethodDescriptor.java:135)
        at com.google.devtools.build.lib.syntax.FuncallExpression.invokeObjectMethod(FuncallExpression.java:930)
        at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:898)
        at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
        at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:201)
        at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
        at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
        at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:475)
        at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:437)
        at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
        at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
        at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
        at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:201)
        at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
        at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
        at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:475)
        at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:437)
        at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
        at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
        at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
        at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:201)
        at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
        at com.google.devtools.build.lib.syntax.Eval.execStatements(Eval.java:231)
        at com.google.devtools.build.lib.syntax.Eval.execIf(Eval.java:144)
        at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:214)
        at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
        at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
        at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:475)
        at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:437)
        at com.google.devtools.build.lib.bazel.repository.skylark.SkylarkRepositoryFunction.fetch(SkylarkRepositoryFunction.java:167)
        at com.google.devtools.build.lib.rules.repository.RepositoryDelegatorFunction.fetchRepository(RepositoryDelegatorFunction.java:288)
        at com.google.devtools.build.lib.rules.repository.RepositoryDelegatorFunction.compute(RepositoryDelegatorFunction.java:215)
        at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:451)
        ... 4 more
java.lang.RuntimeException: Unrecoverable error while evaluating node 'REPOSITORY_DIRECTORY:@local_config_cuda' (requested by nodes 'REPOSITORY:@local_config_cuda')
        at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:528)
        at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:399)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ClassCastException: class com.google.devtools.build.lib.syntax.SkylarkList$MutableList cannot be cast to class java.lang.String (com.google.devtools.build.lib.syntax.SkylarkList$MutableList is in unnamed module of loader 'app'; java.lang.String is in module java.base of loader 'bootstrap')
        at com.google.devtools.build.lib.bazel.repository.skylark.SkylarkRepositoryContext.createFileFromTemplate(SkylarkRepositoryContext.java:295)
        at jdk.internal.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.base/java.lang.reflect.Method.invoke(Unknown Source)
        at com.google.devtools.build.lib.syntax.MethodDescriptor.call(MethodDescriptor.java:135)
        at com.google.devtools.build.lib.syntax.FuncallExpression.invokeObjectMethod(FuncallExpression.java:930)
        at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:898)
        at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
        at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:201)
        at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
        at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
        at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:475)
        at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:437)
        at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
        at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
        at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
        at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:201)
        at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
        at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
        at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:475)
        at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:437)
        at com.google.devtools.build.lib.syntax.FuncallExpression.callFunction(FuncallExpression.java:992)
        at com.google.devtools.build.lib.syntax.FuncallExpression.doEval(FuncallExpression.java:904)
        at com.google.devtools.build.lib.syntax.Expression.eval(Expression.java:75)
        at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:201)
        at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
        at com.google.devtools.build.lib.syntax.Eval.execStatements(Eval.java:231)
        at com.google.devtools.build.lib.syntax.Eval.execIf(Eval.java:144)
        at com.google.devtools.build.lib.syntax.Eval.execDispatch(Eval.java:214)
        at com.google.devtools.build.lib.syntax.Eval.exec(Eval.java:183)
        at com.google.devtools.build.lib.syntax.UserDefinedFunction.call(UserDefinedFunction.java:91)
        at com.google.devtools.build.lib.syntax.BaseFunction.callWithArgArray(BaseFunction.java:475)
        at com.google.devtools.build.lib.syntax.BaseFunction.call(BaseFunction.java:437)
        at com.google.devtools.build.lib.bazel.repository.skylark.SkylarkRepositoryFunction.fetch(SkylarkRepositoryFunction.java:167)
        at com.google.devtools.build.lib.rules.repository.RepositoryDelegatorFunction.fetchRepository(RepositoryDelegatorFunction.java:288)
FAILED: Build did NOT complete successfully (0 packages loaded)
    currently loading: tensorflow/core/kernels
    Fetching @local_config_cuda; fetching

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 25 (8 by maintainers)

Most upvoted comments

Hi @chengleiwang !Could you refer this link for suitable bazel and tensorflow versions. Could you try with tensorflow_gpu 2.0 or 2.1 ? Please open a new issue in case you face any errors, we will get you the right help .Thanks!

mohantym on Sep 3, 2021