tensorflow: Can't quantize nodes of an RNN ("The node has inputs from different frames.")
What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?
I found a few similar, but not equivalent, problems with frozen/quantized graphs:
Cannot import graph_def for 8-bit Quantized cnn model #5470
tf.import_graph_def: graph_def is invalid at node #4044
Unable to import frozen graph with batchnorm #3628
Environment info
Operating System: Ubuntu 16.10
Installed version of CUDA and cuDNN:
(please attach the output of ls -l /path/to/cuda/lib/libcud*
):
-rw-r--r-- 1 root root 558720 Sep 14 20:02 /usr/local/cuda/lib64/libcudadevrt.a
lrwxrwxrwx 1 root root 16 Sep 14 20:05 /usr/local/cuda/lib64/libcudart.so -> libcudart.so.8.0
lrwxrwxrwx 1 root root 19 Sep 14 20:05 /usr/local/cuda/lib64/libcudart.so.8.0 -> libcudart.so.8.0.44
-rw-r--r-- 1 root root 415432 Sep 14 20:02 /usr/local/cuda/lib64/libcudart.so.8.0.44
-rw-r--r-- 1 root root 775162 Sep 14 20:02 /usr/local/cuda/lib64/libcudart_static.a
lrwxrwxrwx 1 root root 13 Jan 27 22:01 /usr/local/cuda/lib64/libcudnn.so -> libcudnn.so.5
lrwxrwxrwx 1 root root 17 Jan 27 22:01 /usr/local/cuda/lib64/libcudnn.so.5 -> libcudnn.so.5.1.5
-rwxr-xr-x 1 root root 79337624 Jan 27 22:01 /usr/local/cuda/lib64/libcudnn.so.5.1.5
-rw-r--r-- 1 root root 69756172 Jan 27 22:01 /usr/local/cuda/lib64/libcudnn_static.a
If installed from binary pip package, provide:
- A link to the pip package you installed: https://pypi.python.org/pypi/tensorflow-gpu
- The output from
python -c "import tensorflow; print(tensorflow.__version__)"
:
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally
0.12.1
If possible, provide a minimal reproducible example (We usually don’t have time to read hundreds of lines of your code)
I reduced this to a simple RNN model available here on branch tf_issue_7162: https://github.com/reuben/tf-export-test/tree/tf_issue_7162
You don’t have to train it again, the repository includes a checkpoint with trained weights. If you run the commands starting from the freeze_graph in the README there, you get this error when importing the quantized graph:
Traceback (most recent call last):
File "import.py", line 14, in <module>
imports = tf.import_graph_def(pb, name="")
File "/home/reuben/.local/lib/python2.7/site-packages/tensorflow/python/framework/importer.py", line 339, in import_graph_def
% (input_name,)))
ValueError: graph_def is invalid at node u'RNN/cond/Greater/y': More inputs specified ('RNN/cond/Switch:0') than the op expects..
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 32 (19 by maintainers)
Commits related to this issue
- Fix quantizing graph with control flows Related to #7162 — committed to wodesuck/tensorflow by wodesuck 7 years ago
- Fix quantizing graph with control flows (#9792) Related to #7162 — committed to tensorflow/tensorflow by wodesuck 7 years ago
I have similar errors when quantize_nodes for Tensor2Tensor Transformer model. It’s no problem after freeze_graph, it’s also no problem with quantize_weights. But when I qutize_nodes, it will:
I have tried TF1.4 and TF1.7, and the error is the same. Can anyone help me?
Looks like this fixes the problem on our network as well. I just tested applying the documented transforms for “8 bits calculations”, and the resulting file does indeed work.
More precisely,
add_default_attributes strip_unused_nodes(type=float, shape="1,299,299,3") remove_nodes(op=Identity, op=CheckNumerics) fold_old_batch_norms quantize_weights quantize_nodes strip_unused_nodes sort_by_execution_order'
produced a network where our “output_node” is removed.But
add_default_attributes strip_unused_nodes(type=float, shape="1,299,299,3") fold_old_batch_norms quantize_weights quantize_nodes strip_unused_nodes sort_by_execution_order'
produces a network where we can do inference.