tensorflow: bazel compiliation is broken! build failure due to github checksums changing

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Arch Linux
TensorFlow installed from (source or binary): Source
TensorFlow version (use command below): master
Python version: Python 3.6.2
Bazel version (if compiling from source): 0.5.4
CUDA/cuDNN version: N/A
GPU model and memory: N/A
Exact command to reproduce: bazel build --verbose_failures //tensorflow/contrib/android:libtensorflow_inference.so --crosstool_top=//external:android/crosstool --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --cpu=armeabi-v7a

Describe the problem

GitHub tarball checksums have changed making it impossible to build tensorflow since the checksums don’t match any more.

https://github.com/bazelbuild/bazel/issues/3722

Source code / logs

ERROR: /home/travis/tensorflow/tensorflow/contrib/android/BUILD:72:1: error loading package 'tensorflow/core': Encountered error while reading extension file 'protobuf.bzl': no such package '@protobuf//': java.io.IOException: Error downloading [https://github.com/google/protobuf/archive/0b059a3d8a8f8aa40dde7bea55edca4ec5dfea66.tar.gz, http://mirror.bazel.build/github.com/google/protobuf/archive/0b059a3d8a8f8aa40dde7bea55edca4ec5dfea66.tar.gz] to /home/travis/.cache/bazel/_bazel_travis/c397b760afc31b444fffb10b0086dea5/external/protobuf/0b059a3d8a8f8aa40dde7bea55edca4ec5dfea66.tar.gz: Checksum was e5fdeee6b28cf6c38d61243adff06628baa434a22b5ebb7432d2a7fbabbdb13d but wanted 6d43b9d223ce09e5d4ce8b0060cb8a7513577a35a64c7e3dad10f0703bf3ad93 and referenced by '//tensorflow/contrib/android:libtensorflow_inference.so'

 /tmp/foo  curl -L https://github.com/google/protobuf/archive/0b059a3d8a8f8aa40dde7bea55edca4ec5dfea66.tar.gz | sha256sum
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   157    0   157    0     0    157      0 --:--:-- --:--:-- --:--:--   301
100 4274k  100 4274k    0     0  4274k      0  0:00:01  0:00:01 --:--:-- 8710k
e5fdeee6b28cf6c38d61243adff06628baa434a22b5ebb7432d2a7fbabbdb13d  -
 /tmp/foo  curl http://mirror.bazel.build/github.com/google/protobuf/archive/0b059a3d8a8f8aa40dde7bea55edca4ec5dfea66.tar.gz | sha256sum
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4274k  100 4274k    0     0  4274k      0  0:00:01 --:--:--  0:00:01 6177k
6d43b9d223ce09e5d4ce8b0060cb8a7513577a35a64c7e3dad10f0703bf3ad93  -

About this issue

Original URL
State: closed
Created 7 years ago
Reactions: 30
Comments: 17 (10 by maintainers)

Commits related to this issue

Remove github mirror of protobuf due to mismatched sha256 sum. Resolves #12979 PiperOrigin-RevId: 168723964 — committed to drpngx/tensorflow by andrewharp 7 years ago
Fix bug regarding invalid sha checksums in tensorflow.bzl, See https://github.com/tensorflow/tensorflow/issues/12979. — committed to tensorflow/fold by delesley 7 years ago
https://github.com/tensorflow/tensorflow/issues/12979: patch broken sha in workspace.bzl for protobuf dep until — committed to mentalityai/tensorflow by deleted user 7 years ago
Remove github mirror of protobuf due to mismatched sha256 sum. Resolves #12979 PiperOrigin-RevId: 168723964 — committed to codrut3/tensorflow by andrewharp 7 years ago
Applied hotfix https://github.com/tensorflow/tensorflow/issues/12979#issuecomment-328671648 — committed to esmanchik/tensorflow by esmanchik 7 years ago
commons: Apply a temporary fix for tensorflow/tensorflow#12979 — committed to lablup/backend.ai-kernels by achimnol 7 years ago
Update workspace.bzl https://github.com/tensorflow/tensorflow/issues/12979 try removing github mirror rather than changing the sah256. it seems the tarball has wrong sha — committed to LogiVideo/tensorflow by deleted user 7 years ago

Most upvoted comments

Temporary fix:

sed -i '\@https://github.com/google/protobuf/archive/0b059a3d8a8f8aa40dde7bea55edca4ec5dfea66.tar.gz@d' tensorflow/workspace.bzl

+56

d4l3k on Sep 11, 2017

@tlc I added the following (temporary) line to one of my Dockerfiles. Note that this completely disables checksum validation, so this is probably a really stupid idea — you have been warned…

sed -ri "/^\W+sha256 = \"[^\"]+\"\W+$/d" tensorflow/workspace.bzl

+18

thomasjo on Sep 18, 2017

There should probably be a new tensorflow release with those fixes since currently no one can build the latest release.

svenstaro on Sep 16, 2017

As a workaround just comment the sha256 checksum lines in tensorflow/workspace.bzl. The checksums changed probably due to some library change on GitHub side.

bzamecnik on Sep 12, 2017

Answer to myself: https://github.com/libgit2/libgit2/issues/4343#issuecomment-328631745 TL;DR it does confirm that Github changes the code to produce tarball, and that the way they are used is fundamentaly risky.

lissyx on Sep 13, 2017

Would anyone happen to know if it’s an expected behavior from Github? What is going to happen for the tarball hosted on the mirror.bazel.build, are they going to be updated ? Should we trust that new sha256 ? I have spotted that the URL I use for RPi3 toolchain download is impacted as well:

ERROR: /home/build-user/DeepSpeech/tf/tools/arm_compiler/BUILD:116:1: no such package '@GccArmRpi//': Error downloading [https://github.com/raspberrypi/tools/archive/0e906ebc527eab1cdbf7adabff5b474da9562e9f.tar.gz] to /home/build-user/.cache/bazel/_bazel_build-user/c049635af10109d54fe54c6ebd9031b2/external/GccArmRpi/0e906ebc527eab1cdbf7adabff5b474da9562e9f.tar.gz: Checksum was 4c622a5c7b9feb9615d4723b03a13142a7f3f813f9296861d5401282b9fbea96 but wanted 970285762565c7890c6c087d262b0a18286e7d0384f13a37786d8521773bc969 and referenced by '//tools/arm_compiler:gcc_linux_linker_files'.

lissyx on Sep 12, 2017