tensorflow: OutOfRangeError/Early EOF on file read in Windows Server 2012
Hi folks,
I am finding that tensorflow code snippets which ran on Windows 10 are failing on Windows Server 2012/NTFS. In particular, anytime I try to load a file (with tf.gfile.Open, tf.gfile.FastGFile, or tf.contrib.slim.assign_from_checkpoint_fn), I encounter an “out of range” error.
What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?
I found Git issue #6791 (Contrib support for Windows) where it was suggested that the fault lies with contrib packages. However, the code sample I provide below displays the problem without using any contrib imports.
Environment info
Operating System: Windows Server 2012 R2
Installed version of CUDA and cuDNN: N/A
(please attach the output of ls -l /path/to/cuda/lib/libcud*):
If installed from binary pip package, provide:
- A link to the pip package you installed: tensorflow-0.12.1-cp35-cp35m-win_amd64.whl
- The output from
python -c "import tensorflow; print(tensorflow.__version__)". 0.12.1
If possible, provide a minimal reproducible example (We usually don’t have time to read hundreds of lines of your code)
with tf.Graph().as_default():
with tf.Session('') as sess:
image_data = tf.gfile.FastGFile(filepath_to_image, 'r').read()
---------------------------------------------------------------------------
OutOfRangeError Traceback (most recent call last)
<ipython-input-11-a7847d42df75> in <module>()
1 with tf.Graph().as_default():
2 with tf.Session('') as sess:
----> 3 image_data = tf.gfile.FastGFile(training_filenames[0], 'r').read()
C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\lib\io\file_io.py in read(self, n)
110 else:
111 length = n
--> 112 return pywrap_tensorflow.ReadFromStream(self._read_buf, length, status)
113
114 def seek(self, position):
C:\Anaconda\envs\py35\lib\contextlib.py in __exit__(self, type, value, traceback)
64 if type is None:
65 try:
---> 66 next(self.gen)
67 except StopIteration:
68 return
C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\framework\errors_impl.py in raise_exception_on_not_ok_status()
467 None, None,
468 compat.as_text(pywrap_tensorflow.TF_Message(status)),
--> 469 pywrap_tensorflow.TF_GetCode(status))
470 finally:
471 pywrap_tensorflow.TF_DeleteStatus(status)
OutOfRangeError: reached end of file
A similar error is encountered when reading a text file, and a more complicated DataLossError occurs when trying to load a model from a checkpoint (but the trace includes references to “out of range” errors, so I assume the same underlying problem is responsible).
The same code snippet gives no error and returns the image byte data when run on the same version of Tensorflow on Windows 10 (also NTFS).
What other attempted solutions have you tried?
I can workaround this problem for images and text files by using native Python to read the files. However, I really need to use tensorflow built-ins to e.g. read a trained model from a checkpoint.
Logs or other output that would be helpful
(If logs are large, please upload as attachment or provide link).
See above for output of MWE.
Thanks very much for your help!
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 20 (8 by maintainers)
@oldmonk101 This bug (
tell()raising anAttributeErrorif called beforeread()orwrite()) was present in all versions of TensorFlow up to 1.0.1, but is fixed in 1.1. Upgrading to the 1.1 release candidate or nightly version will fix the problem.spend some time on this: @smartdolphin I can reproduce the same exception on 0.12.1 but on 1.0 this is gone. Could you update to 1.0 (pip install --upgrade tensorflow) and see if this works for you? @mawah, I can’t reproduce your problem. I’ll send you some email if we can debug this in your setup.
I remember seeing above gzip trace above back in October and there was some discussion about eof semantic in ReadFromStream(). Just tried my old test app again and it fails, but seem to work linux. I can debug this a little … might take a day or 2 before I get to it,