tensorflow: TocoConverter: permute layer after dim-reducing reshape / squeeze: error on conversion

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: -
• TensorFlow installed from (source or binary): binary
• TensorFlow version (use command below): v1.10.1-0-g4dcfddc5d1 1.10.1
• Python version: 3.5.5
• Bazel version (if compiling from source): -
• GCC/Compiler version (if compiling from source): -
• CUDA/cuDNN version: V9.0.176
• GPU model and memory: NVIDIA TITAN Xp, 12196MiB
• Exact command to reproduce: see minimum example code below

Describe the problem

The code example below containing a model with an input, “transformation” and permute layer compiles fine on pc and it is also possible to call predict()on it and getting the expected results. The input layer expects the last dimension to be 1, the transform layer being either a reshape or a squeeze layer gets rid of the 1, and the permutation layer just pushes the first dimension after the batch size to the end. The native Keras reshape layer (A) is the only one that doesn’t work because it doesn’t support dimensionality reduction. Its output shape would be (None, None, 10, 42). However, lines (B), © and (D) all work fine and it doesn’t matter which one is commented in. In the same way it doesn’t matter whether the Keras permute layer (E) or the backend version (F) is commented in.

If all transform layers (A-D) are commented out and only one permute layer (E) or (F) is commented in, directly gets passed input and an input shape of (2,3,1) for (E) or (0,2,3,1) for (F), the model compiles and the TocoConverter produces a tflite-file. If all permute layers (E, F) are commented out and only one transform layer (B) or © or (D) is commented in and the outputs of Model is set to [transform], the model also compiles and the TocoConverter produces a tflite-file.

But for each combination of (B-C) and (E, F) being commented in so that one transform layer and one permute layer is there, the TocoConverter doesn’t produce a tflite-file although the model compiles and the model summary looks as it should.

The expected result is the minimum example code running without errors and producing a tflite model file from the converted keras model file.

---

Edit: For the time being and for everyone with the same problem, I wrote an updated minimum example code with an “emulated” permute layer as a workaround. It uses split, concatenate and reshape operations and you find it at the end of this page after the tracebacks.

---

Source code / logs

Minimum Example Code:

from tensorflow.keras import Model
from tensorflow.keras.models import save_model

from tensorflow.keras.layers import Input, Lambda, Permute, Reshape
from tensorflow.keras.backend import permute_dimensions
from tensorflow.keras.backend import squeeze as b_squeeze
from tensorflow import reshape, squeeze

from tensorflow.contrib import lite


input = Input(shape=(10, 42, 1))

# transform = Reshape((-1, 10, 42))(input)					# (A)
transform = Lambda(lambda x: reshape(x, (-1, 10, 42)))(input)			# (B)
# transform = Lambda(lambda x: squeeze(x, axis=-1))(input)			# (C)
# transform = Lambda(lambda x: b_squeeze(x, axis=-1))(input)			# (D)

permute = Permute((2,1))(transform)						# (E)
# permute = Lambda(lambda x: permute_dimensions(x, (0,2,1)))(transform)		# (F)

keras_model = Model(inputs=[input], outputs=[permute])
keras_model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])


print(keras_model.summary())
save_model(model=keras_model, filepath="keras_model.hdf5",
	overwrite=True, include_optimizer=True)

converter = lite.TocoConverter.from_keras_model_file("keras_model.hdf5")
tflite_model = converter.convert()
with open("tflite_model.tflite", "wb") as f: f.write(tflite_model)

keras_model.summary() for each of the valid six combinations:

_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         (None, 10, 42, 1)         0
_________________________________________________________________
lambda (Lambda)              (None, 10, 42)            0
_________________________________________________________________
permute (Permute)            (None, 42, 10)            0
=================================================================
Total params: 0
Trainable params: 0
Non-trainable params: 0
_________________________________________________________________

Traceback for (B)+(E) and (B)+(F):

Traceback (most recent call last):
  File "minimal.py", line 32, in <module>
    tflite_model = converter.convert()
  File "/usr/stud/staab/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/contrib/lite/python/lite.py", line 374, in convert
    dump_graphviz_video=self.dump_graphviz_video)
  File "/usr/stud/staab/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/contrib/lite/python/convert.py", line 246, in toco_convert
    input_data.SerializeToString())
  File "/usr/stud/staab/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/contrib/lite/python/convert.py", line 106, in toco_convert_protos
    (stdout, stderr))
RuntimeError: TOCO failed see console for info.
b'2018-09-06 10:40:25.228797: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 2 operators, 5 arrays (0 quantized)\n
2018-09-06 10:40:25.228949: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 2 operators, 5 arrays (0 quantized)\n
2018-09-06 10:40:25.229050: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After general graph transformations pass 1: 2 operators, 5 arrays (0 quantized)\n
2018-09-06 10:40:25.229088: F tensorflow/contrib/lite/toco/graph_transformations/propagate_fixed_sizes.cc:1402] Check failed: axis < input_shape.dimensions_count() (1211997096 vs. 4)\n
Aborted (core dumped)\n'
None

The number 1211997096 is always randomly different and even can be negative.

Traceback for ©+(E), (D)+(E), ©+(F) and (D)+(F):

Traceback (most recent call last):
  File "minimal.py", line 32, in <module>
    tflite_model = converter.convert()
  File "/usr/stud/staab/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/contrib/lite/python/lite.py", line 374, in convert
    dump_graphviz_video=self.dump_graphviz_video)
  File "/usr/stud/staab/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/contrib/lite/python/convert.py", line 246, in toco_convert
    input_data.SerializeToString())
  File "/usr/stud/staab/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/contrib/lite/python/convert.py", line 106, in toco_convert_protos
    (stdout, stderr))
RuntimeError: TOCO failed see console for info.
b'2018-09-06 10:44:46.550688: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 2 operators, 4 arrays (0 quantized)\n
2018-09-06 10:44:46.550805: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 2 operators, 4 arrays (0 quantized)\n
2018-09-06 10:44:46.550870: F tensorflow/contrib/lite/toco/graph_transformations/propagate_fixed_sizes.cc:1395] Check failed: perm.size() == input_shape.dimensions_count() (3 vs. 4)Transpose permutation input permute/transpose/perm must be same length as input dimensions\n
Aborted (core dumped)\n'
None

Minimum Example Code with Workaround:

from tensorflow.keras import Model
from tensorflow.keras.models import save_model

from tensorflow.keras.layers import Input, Lambda, Permute, Reshape, concatenate
from tensorflow.keras.backend import permute_dimensions
from tensorflow.keras.backend import squeeze as b_squeeze
from tensorflow import reshape, squeeze, split, expand_dims

from tensorflow.contrib import lite


input = Input(shape=(10, 42, 1))

# transform = Reshape((-1, 10, 42))(input)					# (A)
transform = Lambda(lambda x: reshape(x, (-1, 10, 42)))(input)			# (B)
# transform = Lambda(lambda x: squeeze(x, axis=-1))(input)			# (C)
# transform = Lambda(lambda x: b_squeeze(x, axis=-1))(input)			# (D)

# Emulated permute to come around a bug: split, concatenate, reshape
#------------------------------------------------------------------------------
split_p = Lambda(lambda x: split(expand_dims(x, axis=-1), num_or_size_splits=10,
                                                            axis=1))(transform)
merge_p = concatenate(split_p)
squeeze_p = Lambda(lambda x: reshape(x, (-1, 42, 10)))(merge_p)
#------------------------------------------------------------------------------

keras_model = Model(inputs=[input], outputs=[squeeze_p])
keras_model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])


print(keras_model.summary())
save_model(model=keras_model, filepath="keras_model.hdf5",
	overwrite=True, include_optimizer=True)

converter = lite.TocoConverter.from_keras_model_file("keras_model.hdf5")
tflite_model = converter.convert()
with open("tflite_model.tflite", "wb") as f: f.write(tflite_model)