tensorflow: OutOfRangeError/Early EOF on file read in Windows Server 2012

Hi folks,

I am finding that tensorflow code snippets which ran on Windows 10 are failing on Windows Server 2012/NTFS. In particular, anytime I try to load a file (with tf.gfile.Open, tf.gfile.FastGFile, or tf.contrib.slim.assign_from_checkpoint_fn), I encounter an “out of range” error.

What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?

I found Git issue #6791 (Contrib support for Windows) where it was suggested that the fault lies with contrib packages. However, the code sample I provide below displays the problem without using any contrib imports.

Environment info

Operating System: Windows Server 2012 R2

Installed version of CUDA and cuDNN: N/A (please attach the output of ls -l /path/to/cuda/lib/libcud*):

If installed from binary pip package, provide:

  1. A link to the pip package you installed: tensorflow-0.12.1-cp35-cp35m-win_amd64.whl
  2. The output from python -c "import tensorflow; print(tensorflow.__version__)". 0.12.1

If possible, provide a minimal reproducible example (We usually don’t have time to read hundreds of lines of your code)

with tf.Graph().as_default():
        with tf.Session('') as sess:
                image_data = tf.gfile.FastGFile(filepath_to_image, 'r').read()
---------------------------------------------------------------------------
OutOfRangeError                           Traceback (most recent call last)
<ipython-input-11-a7847d42df75> in <module>()
      1 with tf.Graph().as_default():
      2     with tf.Session('') as sess:
----> 3         image_data = tf.gfile.FastGFile(training_filenames[0], 'r').read()
C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\lib\io\file_io.py in read(self, n)
    110       else:
    111         length = n
--> 112       return pywrap_tensorflow.ReadFromStream(self._read_buf, length, status)
    113 
    114   def seek(self, position):
C:\Anaconda\envs\py35\lib\contextlib.py in __exit__(self, type, value, traceback)
     64         if type is None:
     65             try:
---> 66                 next(self.gen)
     67             except StopIteration:
     68                 return
C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\framework\errors_impl.py in raise_exception_on_not_ok_status()
    467           None, None,
    468           compat.as_text(pywrap_tensorflow.TF_Message(status)),
--> 469           pywrap_tensorflow.TF_GetCode(status))
    470   finally:
    471     pywrap_tensorflow.TF_DeleteStatus(status)
OutOfRangeError: reached end of file

A similar error is encountered when reading a text file, and a more complicated DataLossError occurs when trying to load a model from a checkpoint (but the trace includes references to “out of range” errors, so I assume the same underlying problem is responsible).

The same code snippet gives no error and returns the image byte data when run on the same version of Tensorflow on Windows 10 (also NTFS).

What other attempted solutions have you tried?

I can workaround this problem for images and text files by using native Python to read the files. However, I really need to use tensorflow built-ins to e.g. read a trained model from a checkpoint.

Logs or other output that would be helpful

(If logs are large, please upload as attachment or provide link).

See above for output of MWE.

Thanks very much for your help!

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 20 (8 by maintainers)

Most upvoted comments

@oldmonk101 This bug (tell() raising an AttributeError if called before read() or write()) was present in all versions of TensorFlow up to 1.0.1, but is fixed in 1.1. Upgrading to the 1.1 release candidate or nightly version will fix the problem.

spend some time on this: @smartdolphin I can reproduce the same exception on 0.12.1 but on 1.0 this is gone. Could you update to 1.0 (pip install --upgrade tensorflow) and see if this works for you? @mawah, I can’t reproduce your problem. I’ll send you some email if we can debug this in your setup.

I remember seeing above gzip trace above back in October and there was some discussion about eof semantic in ReadFromStream(). Just tried my old test app again and it fails, but seem to work linux. I can debug this a little … might take a day or 2 before I get to it,