tensorflow: Initial read on nonexistent tf.gfile.GFile in w+ mode crashes

System information

  • Have I written custom code: Yes
  • OS Platform and Distribution: Linux Ubuntu 18.04
  • TensorFlow installed from: binary
  • TensorFlow version: v1.14.0-rc1-22-gaf24dc9 1.14.0
  • Python version: 3.7

Describe the current behavior Python raises tensorflow.python.framework.errors_impl.NotFoundError when doing a first read (no writes before it) on a nonexistent tf.gfile.GFile in w+ mode.

Describe the expected behavior Read on an empty w+ file should return an empty string. One problem with the current behaviour is that numpy.savez() crashes when writing to a GFile.

Code to reproduce the issue

import tensorflow as tf

with tf.io.gfile.GFile('test.txt', 'w+') as f:
    f.read()

Other info / logs

Traceback (most recent call last):
  File "test_gfile.py", line 5, in <module>
    f.read()
  File "/VENV/lib/python3.7/site-packages/tensorflow/python/lib/io/file_io.py", line 122, in read
    self._preread_check()
  File "/VENV/lib/python3.7/site-packages/tensorflow/python/lib/io/file_io.py", line 84, in _preread_check
    compat.as_bytes(self.__name), 1024 * 512)
tensorflow.python.framework.errors_impl.NotFoundError: test.txt; No such file or directory

About this issue

  • Original URL
  • State: open
  • Created 7 years ago
  • Comments: 18 (6 by maintainers)

Commits related to this issue

Most upvoted comments

Hi,

Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.

The Tensorflow team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.

Please follow the release notes to stay up to date with the latest developments which are happening in the Tensorflow space.

I was able to replicate the issue in tf-nightly 2.12.0-dev20221215. Please find the gist for reference. Thank you.

a similar approach that works for me.

io_buffer = io.BytesIO()
np.savez(io_buffer, ...)
with gfile.Open(path, "wb") as f:
  f.write(io_buffer.getvalue())

from @eirism in https://github.com/tensorflow/tensorflow/issues/32090#issuecomment-526563509:

Workarounds are to either write something, e.g. "", to the file before reading so that the file is created, or manually create the file first as you suggest. But those are workarounds to a problem that I think should be solved in GFile.

could you give an example of such a workaround? I tried the following unsuccessful:

import numpy as np
import tensorflow as tf

with tf.io.gfile.GFile("output.npz", "w") as file:
    file.write("")
    np.savez(file, content=np.array([1, 2, 3]))

Still getting

tensorflow.python.framework.errors_impl.PermissionDeniedError: File isn't open for reading

f.seek() also crashes in the same way as f.read().

Related to the fact that GFile does not truncate files the same way as Python, reading an existing file opened with w+ will return the text in the file instead of "".

Example:

import tensorflow as tf

with open('existing.txt', 'w') as f:
    f.write('txt')
    f.flush()

with tf.io.gfile.GFile('existing.txt', 'w+') as f:
    print(f.read())  # Prints txt, should print ""

test.txt is not in your current working directory. Please, make sure test.txt and your python-file/jupyter-notebook are in same directory.Thanks!