tensorflow: Cannot Read From Google Storage with tf.io.gfile.GFile under intel-tensorflow==1.14.0

Support for Google Storage (gs protocol) seems to be missing from intel-tensorflow==1.14.0, which is unexpected. I’m aware that this is no core tensorflow issue but I’ve seen that some engineers from Intel are active in this repository.

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): docker image intelaipg/intel-optimized-tensorflow:1.14.0-mkl-py3 (Ubuntu 18.04.2 LTS)
  • Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
  • TensorFlow installed from (source or binary): No
  • TensorFlow version (use command below): 1.14.0
  • Python version: 3.6.8
  • Bazel version (if compiling from source): n/a
  • GCC/Compiler version (if compiling from source): n/a
  • CUDA/cuDNN version: n/a
  • GPU model and memory: n/a

You can collect some of this information using our environment capture script You can also obtain the TensorFlow version with:

  1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
  2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

Output: v1.14.0-1-hardened-0-g340d16ee58 1.14.0

Describe the current behavior

Google storage doesn’t seem to be supported for some reason:

> python
Python 3.6.8 (default, Jan 14 2019, 11:02:34)
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf; tf.io.gfile.GFile("gs://some_bucket/test.txt").read()
[deprecation warning redacted]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/lib/io/file_io.py", line 122, in read
    self._preread_check()
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/lib/io/file_io.py", line 84, in _preread_check
    compat.as_bytes(self.__name), 1024 * 512)
tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme 'gs' not implemented (file: 'gs://some_bucket/test.txt')

Describe the expected behavior

Verified to work with 1.13.2:

> python
Python 3.6.8 (default, Jan 14 2019, 11:02:34)
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf; tf.io.gfile.GFile("gs://some_bucket/test.txt").read()
[deprecation warning redacted]
'test\n'

Standalone code to reproduce the issue

See above.

Other info / logs n/a

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 16 (8 by maintainers)

Most upvoted comments

Ok, I’ll close it in favor of #40989. But note that The the Tensorflow build in this image intelaipg/intel-optimized-tensorflow:1.14.0-mkl-py3 does not support GFile and that there’s no update for it.

@pks. Thank you! The TF1.14 is out of maintain. But TF 1.15 will be maintained for more months.