tensorflow: Initial read on nonexistent tf.gfile.GFile in w+ mode crashes

System information

Have I written custom code: Yes
OS Platform and Distribution: Linux Ubuntu 18.04
TensorFlow installed from: binary
TensorFlow version: v1.14.0-rc1-22-gaf24dc9 1.14.0
Python version: 3.7

Describe the current behavior Python raises tensorflow.python.framework.errors_impl.NotFoundError when doing a first read (no writes before it) on a nonexistent tf.gfile.GFile in w+ mode.

Describe the expected behavior Read on an empty w+ file should return an empty string. One problem with the current behaviour is that numpy.savez() crashes when writing to a GFile.

Code to reproduce the issue

import tensorflow as tf

with tf.io.gfile.GFile('test.txt', 'w+') as f:
    f.read()

Other info / logs

Traceback (most recent call last):
  File "test_gfile.py", line 5, in <module>
    f.read()
  File "/VENV/lib/python3.7/site-packages/tensorflow/python/lib/io/file_io.py", line 122, in read
    self._preread_check()
  File "/VENV/lib/python3.7/site-packages/tensorflow/python/lib/io/file_io.py", line 84, in _preread_check
    compat.as_bytes(self.__name), 1024 * 512)
tensorflow.python.framework.errors_impl.NotFoundError: test.txt; No such file or directory

About this issue

Original URL
State: open
Created 5 years ago
Comments: 15 (5 by maintainers)

Commits related to this issue

Fix save memory store using GFile. Writing to a buffer to avoid read error in np.savez when using GFile. See: https://github.com/tensorflow/tensorflow/issues/32090 — committed to tensorflow/similarity by owenvallis 2 years ago
Fix save memory store using GFile. Writing to a buffer to avoid read error in np.savez when using GFile. See: https://github.com/tensorflow/tensorflow/issues/32090 — committed to abeltheo/similarity by owenvallis 2 years ago

Most upvoted comments

I was able to replicate the issue in tf-nightly 2.12.0-dev20221215. Please find the gist for reference. Thank you.

chunduriv on Dec 19, 2022

a similar approach that works for me.

io_buffer = io.BytesIO()
np.savez(io_buffer, ...)
with gfile.Open(path, "wb") as f:
  f.write(io_buffer.getvalue())

cnsgsz on Dec 4, 2021

from @eirism in https://github.com/tensorflow/tensorflow/issues/32090#issuecomment-526563509:

Workarounds are to either write something, e.g. "", to the file before reading so that the file is created, or manually create the file first as you suggest. But those are workarounds to a problem that I think should be solved in GFile.

could you give an example of such a workaround? I tried the following unsuccessful:

import numpy as np
import tensorflow as tf

with tf.io.gfile.GFile("output.npz", "w") as file:
    file.write("")
    np.savez(file, content=np.array([1, 2, 3]))

Still getting

tensorflow.python.framework.errors_impl.PermissionDeniedError: File isn't open for reading

patzm on Oct 27, 2021

f.seek() also crashes in the same way as f.read().

eirism on Aug 30, 2019

Related to the fact that GFile does not truncate files the same way as Python, reading an existing file opened with w+ will return the text in the file instead of "".

Example:

import tensorflow as tf

with open('existing.txt', 'w') as f:
    f.write('txt')
    f.flush()

with tf.io.gfile.GFile('existing.txt', 'w+') as f:
    print(f.read())  # Prints txt, should print ""

eirism on Aug 30, 2019

test.txt is not in your current working directory. Please, make sure test.txt and your python-file/jupyter-notebook are in same directory.Thanks!

ravikyram on Aug 30, 2019