tensorflow: Writing a SavedModel on Windows with long file path fails
System information
- Have I written custom code: Yes, see below.
- OS Platform and Distribution: Windows 10 Enterprise 64Bit (10.0.16299)
- Mobile device: N/A
- TensorFlow installed from: Binary
- TensorFlow version: b’v1.10.0-rc1-19-g656e7a2b34’ 1.10.0
- Python version: 3.5.5
- Bazel version: N/A
- GCC/Compiler version: N/A
- CUDA/cuDNN version: N/A
- GPU model and memory: N/A
- Exact command to reproduce: Run the provided script on Windows
Describe the problem
When exporting a SavedModel to a directory with a rather long path the maximum file path length of Windows (260 chars) gets exceeded while creating temporary variable files and the process fails because it can’t find the file (See attachments for the full log):
NotFoundError (see above for traceback): Failed to create a NewWriteableFile: C:\reallylongpath\barfoobarfoobarfoobarfoobarfoobarfoo\barfoobarfoobarfoobarfoobarfoobarfoo\barfoobarfoobarfoobarfoobarfoobarfoo\barfoobarfoo\variables\variables_temp_cbd5fe79f07a4790a08e7c2adf1ed95b/part-00000-of-00001.data-00000-of-00001.tempstate12448200021531902965 : The system cannot find the path specified.
The usual workaround to prepend '\\\\?\\'
to the path (like described here) doesn’t work (See attachments).
Source code / logs
Code to reproduce:
import tensorflow as tf
# Long export path (should be at least 140 chars long)
export_dir = r'C:\reallylongpath\barfoobarfoobarfoobarfoobarfoobarfoo\barfoobarfoobarfoobarfoobarfoobarfoo\barfoobarfoobarfoobarfoobarfoobarfoo\barfoobarfoo'
print('Export dir lenght: {}'.format(len(export_dir)))
# Build an easy graph with variables
x = tf.constant(2)
w = tf.Variable(3)
y = tf.multiply(x, w)
with tf.Session() as sess:
# Initialize the variable
sess.run(w.initializer)
# Save as a SavedModel
tf.saved_model.simple_save(sess,
export_dir,
inputs={'x': x },
outputs={'y': y })
Logs: Full log: log.txt
Full log with '\\\\?\\'
prefix:
log_prefix_0.txt
Full log with '\\?\'
prefix:
log_prefix_1.txt
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 2
- Comments: 19 (2 by maintainers)
Commits related to this issue
- ci: workaround for https://github.com/tensorflow/tensorflow/issues/22498 — committed to DC-analysis/dclab by paulmueller 4 years ago
Same issue with Windows 10 2004 & TensorFlow 2.1.0.
Enabling the Long Path option in Registry Editor solves the problem.
Anyway is there a solution on Python side to avoid modifying Windows parameters?
As mentioned in the original post, adding the UNC prefix
\\\\?\\
or\\?\
to the “too long path” doesn’t work.What do you mean by “for NFS”? I want to use a local path.
Enable Long File Path
is not located in the NTFS folder anymore but inLocalGroupPolicyFolder > Computer Configuration > Administrative Templates > System > Filesystem
(Source)I checked that the configuration was enabled and restarted the computer again (to be sure) but it still doesn’t work. Did you try it?
(I installed tensorflow via pip in an anaconda environment if this matters)
I was having this problem with Windows 10 1909, Python 3.6, Tensorflow 2.3.0
In my case, enabling the local policy solved the problem