tensorflow: When ryzen build tf_to_gpu_binary.exe failed: error executing command error

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template

System information

  • OS : Windows 10 20H2
  • TensorFlow version: 2.4.1
  • Python version: python=3.9
  • Building anaconda env
  • Bazel version : using bazelisk
  • CUDA/cuDNN version: cuda 11.1, cuDNN 8.0.5
  • GPU model and memory: RTX2080, 8G

When i build tensorflow in 10900k build succeed. But error is occur when i try to build ryzen 5600x or ryzen3700x.

INFO: Found 1 target… INFO: Deleting stale sandbox base C:/users/user/_bazel_user/d3ty3xtx/sandbox ERROR: C:/users/user/documents/cpplibrarys/tensorflow/tensorflow/core/kernels/mlir_generated/BUILD:149:1: compile tensorflow/core/kernels/mlir_generated/abs_i64_kernel_cubin.sm_75.bin failed (Exit 1): tf_to_gpu_binary.exe failed: error executing command cd C:/users/user/_bazel_user/d3ty3xtx/execroot/org_tensorflow bazel-out/x64_windows-opt/bin/tensorflow/compiler/mlir/tools/kernel_gen/tf_to_gpu_binary.exe --same_shape=0,1 --unroll_factors=4 --tile_sizes=256 --arch=sm_75 --input=bazel-out/x64_windows-opt/bin/tensorflow/core/kernels/mlir_generated/abs_i64.mlir --output=bazel-out/x64_windows-opt/bin/tensorflow/core/kernels/mlir_generated/abs_i64_kernel_cubin.sm_75.bin Execution platform: @local_execution_config_platform//:platform 2021-01-26 17:38:55.379641: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:194] disabling MLIR crash reproducer, set env var MLIR_CRASH_REPRODUCER_DIRECTORY to enable. warning: Linking two modules of different data layouts: ‘C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1/nvvm/libdevice/libdevice.10.bc’ is ‘e-i64:64-v16:16-v32:32-n16:32:64’ whereas ‘acme’ is ‘e-i64:64-i128:128-v16:16-v32:32-n16:32:64’ 2021-01-26 17:38:55.768832: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code: 0 2021-01-26 17:38:55.845907: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code: 4294967295 2021-01-26 17:38:55.847744: E tensorflow/compiler/mlir/tools/kernel_gen/tf_to_gpu_binary.cc:97] Internal: Lowering to LLVM IR failed. Target //tensorflow/tools/pip_package:build_pip_package failed to build ERROR: C:/users/user/documents/cpplibrarys/tensorflow/tensorflow/python/data/experimental/service/BUILD:11:1 compile tensorflow/core/kernels/mlir_generated/tanh_f32_kernel_cubin.sm_75.bin failed (Exit 1): tf_to_gpu_binary.exe failed: error executing command cd C:/users/user/_bazel_user/d3ty3xtx/execroot/org_tensorflow bazel-out/x64_windows-opt/bin/tensorflow/compiler/mlir/tools/kernel_gen/tf_to_gpu_binary.exe --same_shape=0,1 --unroll_factors=4 --tile_sizes=256 --arch=sm_75 --input=bazel-out/x64_windows-opt/bin/tensorflow/core/kernels/mlir_generated/tanh_f32.mlir --output=bazel-out/x64_windows-opt/bin/tensorflow/core/kernels/mlir_generated/tanh_f32_kernel_cubin.sm_75.bin Execution platform: @local_execution_config_platform//:platform INFO: Elapsed time: 69.840s, Critical Path: 24.63s INFO: 0 processes. FAILED: Build did NOT complete successfully

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 16 (4 by maintainers)

Most upvoted comments

Oh @sherhut you right! Build completed successfully when I pass the --define=tensorflow_enable_mlir_generated_gpu_kernels=0 directly in the bazel invocation! like bazel build --config=opt --config=cuda --define=tensorflow_enable_mlir_generated_gpu_kernels=0 --define=no_tensorflow_py_deps=true //tensorflow/tools/pip_package:build_pip_package

Oh thank you! 😍😍