tensorflow: Can't quantize nodes of an RNN ("The node has inputs from different frames.")

What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?

I found a few similar, but not equivalent, problems with frozen/quantized graphs:

Cannot import graph_def for 8-bit Quantized cnn model #5470

tf.import_graph_def: graph_def is invalid at node #4044

Unable to import frozen graph with batchnorm #3628

Environment info

Operating System: Ubuntu 16.10

Installed version of CUDA and cuDNN: (please attach the output of ls -l /path/to/cuda/lib/libcud*):

-rw-r--r-- 1 root root   558720 Sep 14 20:02 /usr/local/cuda/lib64/libcudadevrt.a
lrwxrwxrwx 1 root root       16 Sep 14 20:05 /usr/local/cuda/lib64/libcudart.so -> libcudart.so.8.0
lrwxrwxrwx 1 root root       19 Sep 14 20:05 /usr/local/cuda/lib64/libcudart.so.8.0 -> libcudart.so.8.0.44
-rw-r--r-- 1 root root   415432 Sep 14 20:02 /usr/local/cuda/lib64/libcudart.so.8.0.44
-rw-r--r-- 1 root root   775162 Sep 14 20:02 /usr/local/cuda/lib64/libcudart_static.a
lrwxrwxrwx 1 root root       13 Jan 27 22:01 /usr/local/cuda/lib64/libcudnn.so -> libcudnn.so.5
lrwxrwxrwx 1 root root       17 Jan 27 22:01 /usr/local/cuda/lib64/libcudnn.so.5 -> libcudnn.so.5.1.5
-rwxr-xr-x 1 root root 79337624 Jan 27 22:01 /usr/local/cuda/lib64/libcudnn.so.5.1.5
-rw-r--r-- 1 root root 69756172 Jan 27 22:01 /usr/local/cuda/lib64/libcudnn_static.a

If installed from binary pip package, provide:

  1. A link to the pip package you installed: https://pypi.python.org/pypi/tensorflow-gpu
  2. The output from python -c "import tensorflow; print(tensorflow.__version__)":
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally
0.12.1

If possible, provide a minimal reproducible example (We usually don’t have time to read hundreds of lines of your code)

I reduced this to a simple RNN model available here on branch tf_issue_7162: https://github.com/reuben/tf-export-test/tree/tf_issue_7162

You don’t have to train it again, the repository includes a checkpoint with trained weights. If you run the commands starting from the freeze_graph in the README there, you get this error when importing the quantized graph:

Traceback (most recent call last):
  File "import.py", line 14, in <module>
    imports = tf.import_graph_def(pb, name="")
  File "/home/reuben/.local/lib/python2.7/site-packages/tensorflow/python/framework/importer.py", line 339, in import_graph_def
    % (input_name,)))
ValueError: graph_def is invalid at node u'RNN/cond/Greater/y': More inputs specified ('RNN/cond/Switch:0') than the op expects..

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 32 (19 by maintainers)

Commits related to this issue

Most upvoted comments

I have similar errors when quantize_nodes for Tensor2Tensor Transformer model. It’s no problem after freeze_graph, it’s also no problem with quantize_weights. But when I qutize_nodes, it will:

tensorflow.python.framework.errors_impl.InvalidArgumentError: The node 'transformer/while/body/parallel_0/body/decoder/layer_prepostprocess/layer_norm/add_1/eightbit' has inputs from different frames. The input 'transformer/while/body/parallel_0/body/decoder/layer_prepostprocess/layer_norm/add_1_eightbit/transformer/while/body/parallel_0/body/decoder/layer_prepostprocess/layer_norm/add_1/Enter/quantize' is in frame 'transformer/while/while_context'. The input 'transformer/while/body/parallel_0/body/decoder/layer_prepostprocess/layer_norm/mul_1/eightbit/requantize' is in frame ''.

I have tried TF1.4 and TF1.7, and the error is the same. Can anyone help me?

Looks like this fixes the problem on our network as well. I just tested applying the documented transforms for “8 bits calculations”, and the resulting file does indeed work.

More precisely, add_default_attributes strip_unused_nodes(type=float, shape="1,299,299,3") remove_nodes(op=Identity, op=CheckNumerics) fold_old_batch_norms quantize_weights quantize_nodes strip_unused_nodes sort_by_execution_order' produced a network where our “output_node” is removed.

But add_default_attributes strip_unused_nodes(type=float, shape="1,299,299,3") fold_old_batch_norms quantize_weights quantize_nodes strip_unused_nodes sort_by_execution_order' produces a network where we can do inference.