gap_sdk: NNTOOL import tflite model error

Sigmoid is supported. I think you were hitting the same regression that you hit with tanh. That graph converts, adjusts, fuses and generates fine now. The fix will be pushed to github tonight I think. I will check.

Acknowledged. It’s a regression introduced by constant folding in the latest release. We will resolve and push a fix.

OK. It should be fixed in the 3.8.0_dev branch. It turned out to be a too general gitignore rule in the sdk root so all regressions tests pass and the sdk integration tests had no network with a sigmoid in it! Go figure. Sorry for the delay.

Hi. We appear to have a very weird git issue between myself and the person looking after Github updates. There is a file missing. We are checking what is going on.

Hi @sousoux, I downloaded the last release but I get the same error. Do you have any idea why?

Tanh was a temporary solution to replace sigmoid because when I tried to import my .tflite model cae_q.zip into NNTOOL, it gives me this error:

(NNT) open cae_q.tflite -q
open - opening graph file cae_q.tflite
tflite - Importing TFLITE model version 3
strided_slice - reducing STRIDED_SLICE_0_12 to a constant
pack - reducing PACK_0_13 to a constant
strided_slice - reducing STRIDED_SLICE_0_18 to a constant
math_mixin - reducing MUL_0_19 to a constant
Traceback (most recent call last):
  File "/home/manucar/.local/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/manucar/.local/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/manucar/.local/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/manucar/gap_sdk/tools/nntool/interpreter/commands/open.py", line 98, in do_open
    self.__open_graph(args.nnfile, args.tensor_file, args.load_quantization, args.load_dequantized)
  File "/home/manucar/gap_sdk/tools/nntool/interpreter/commands/open.py", line 71, in __open_graph
    G = create_graph(graph_file, opts=opts)
  File "/home/manucar/gap_sdk/tools/nntool/importer/importer.py", line 53, in create_graph
    graph = importer.create_graph(filename, opts)
  File "/home/manucar/gap_sdk/tools/nntool/importer/tflite2/tflite.py", line 73, in create_graph
    self._import_tflite_graph(G, TFLiteGraph.from_model(model, 0), opts)
  File "/home/manucar/gap_sdk/tools/nntool/importer/tflite2/tflite.py", line 92, in _import_tflite_graph
    self._import_nodes(G, graph, handlers, all_nodes, self._provisional_outputs, opts)
  File "/home/manucar/gap_sdk/tools/nntool/importer/tflite2/tflite.py", line 182, in _import_nodes
    raise ValueError("no handler found for %s" % node.op_type)
ValueError: no handler found for LOGISTIC
EXCEPTION of type 'ValueError' occurred with message: 'no handler found for LOGISTIC'

So I thought sigmoid activation function was not yet supported, am I wrong?

I’ve fixed that problem and will get it pushed to Github shortly but there is an outstanding issue. We do not have tanh implemented in the fused convolution generator. Tanh is a pretty inefficient activation if you want something very accurate. A min(1, max(-1, v)) approximates it pretty well but obviously not perfectly. We call that a htanh. We could add support for htanh fairly easily or you could experiment with another activation.

For the moment i have switched to tanh activation function and it does load into NNTOOL, i have also quantized the .tflite model to fully 8 bit integer (except the input and output) and i get this other error while giving “fusions --scale8” command:

(NNT) open tan_q32.tflite -q
open - opening graph file tan_q32.tflite
tflite - Importing TFLITE model version 3
strided_slice - reducing STRIDED_SLICE_0_12 to a constant
pack - reducing PACK_0_13 to a constant
strided_slice - reducing STRIDED_SLICE_0_16 to a constant
math_mixin - reducing MUL_0_17 to a constant
nngraph - update graph dimensions
set_aliases - looking for aliased edges
nngraph - calculate liveness
nngraph - update graph dimensions
set_aliases - looking for aliased edges
nngraph - calculate liveness
(NNT tan_q32.tflite 0) adjust
adjust_order - adding transposes to correct tensor order for AT kernels
set_aliases - looking for aliased edges
eliminate_transposes - eliminating unnecessary transposes
eliminate_transposes - search for transposes
eliminate_transposes - ++ Starting up from CONV_2D_0_1[0]
eliminate_transposes - looking up at QUANTIZE_0_0[0] transpose [2, 0, 1]
eliminate_transposes - looking up at input_1[0] transpose [2, 0, 1]
eliminate_transposes - accepted input_1 - input without fixed order - transpose input [2, 0, 1]
eliminate_transposes - ++ Found results for CONV_2D_0_1[0]
eliminate_transposes - ++ Starting down from CONV_2D_0_1[0]
eliminate_transposes - looking down at CONV_2D_0_1_activation[0] transpose [1, 2, 0]
eliminate_transposes - looking down at MUL_0_2[0] transpose [1, 2, 0]
eliminate_transposes - accepted MUL_0_2 - propagated transpose to broadcasted linear op
eliminate_transposes - ++ Found results for CONV_2D_0_1[0]
eliminate_transposes - ++ Starting up from CONV_2D_0_4[0]
eliminate_transposes - looking up at ADD_0_3[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_3 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_4
eliminate_transposes - ++ Starting down from CONV_2D_0_4[0]
eliminate_transposes - looking down at CONV_2D_0_4_activation[0] transpose [1, 2, 0]
eliminate_transposes - looking down at MUL_0_5[0] transpose [1, 2, 0]
eliminate_transposes - accepted MUL_0_5 - propagated transpose to broadcasted linear op
eliminate_transposes - ++ Found results for CONV_2D_0_4[0]
eliminate_transposes - ++ Starting up from MAX_POOL_2D_0_7[0]
eliminate_transposes - looking up at ADD_0_6[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_6 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from MAX_POOL_2D_0_7
eliminate_transposes - ++ Starting down from MAX_POOL_2D_0_7[0]
eliminate_transposes - looking down at RESHAPE_0_8[0] transpose [1, 2, 0]
eliminate_transposes - reshape RESHAPE_0_8 trans: old [1, 2, 0] new None shape: old 8x4x32 new 1024
eliminate_transposes - rejected RESHAPE_0_8 - transpose out - does not reverse
eliminate_transposes - Rejected down edges from MAX_POOL_2D_0_7[0]
eliminate_transposes - ++ Starting up from CONV_2D_0_19[0]
eliminate_transposes - looking up at RESIZE_NEAREST_NEIGHBOR_0_18[0] transpose [2, 0, 1]
eliminate_transposes - rejected RESIZE_NEAREST_NEIGHBOR_0_18 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_19
eliminate_transposes - ++ Starting down from CONV_2D_0_19[0]
eliminate_transposes - looking down at CONV_2D_0_19_activation[0] transpose [1, 2, 0]
eliminate_transposes - looking down at MUL_0_20[0] transpose [1, 2, 0]
eliminate_transposes - accepted MUL_0_20 - propagated transpose to broadcasted linear op
eliminate_transposes - ++ Found results for CONV_2D_0_19[0]
eliminate_transposes - ++ Starting up from CONV_2D_0_22[0]
eliminate_transposes - looking up at ADD_0_21[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_21 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_22
eliminate_transposes - ++ Starting down from CONV_2D_0_22[0]
eliminate_transposes - looking down at CONV_2D_0_22_activation[0] transpose [1, 2, 0]
eliminate_transposes - looking down at MUL_0_23[0] transpose [1, 2, 0]
eliminate_transposes - accepted MUL_0_23 - propagated transpose to broadcasted linear op
eliminate_transposes - ++ Found results for CONV_2D_0_22[0]
eliminate_transposes - ++ Starting up from CONV_2D_0_25[0]
eliminate_transposes - looking up at ADD_0_24[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_24 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_25
eliminate_transposes - ++ Starting down from CONV_2D_0_25[0]
eliminate_transposes - looking down at TANH_0_26[0] transpose [1, 2, 0]
eliminate_transposes - looking down at DEQUANTIZE_0_27[0] transpose [1, 2, 0]
eliminate_transposes - looking down at output_1[0] transpose [1, 2, 0]
eliminate_transposes - accepted output_1 - output without fixed order - transpose output [1, 2, 0]
eliminate_transposes - ++ Found results for CONV_2D_0_25[0]
eliminate_transposes - eliminate transposes
eliminate_transposes_actions - Start Action (up): CONV_2D_0_1
eliminate_transposes_actions - CONV_2D_0_1 delete transpose in[0]
eliminate_transposes_actions - CONV_2D_0_1 set hint in[0] to ['c', 'h', 'w']
eliminate_transposes_actions - input_1 input dims with (2, 0, 1)
eliminate_transposes_actions - End Action (up): input_1
eliminate_transposes_actions - Start Action (down): CONV_2D_0_1
eliminate_transposes_actions - CONV_2D_0_1 delete transpose out[0]
eliminate_transposes_actions - CONV_2D_0_1 set hint out[0] to ['c', 'h', 'w']
eliminate_transposes_actions - MUL_0_2 set transpose in[0] to (1, 2, 0)
eliminate_transposes_actions - End Action (down): MUL_0_2
eliminate_transposes_actions - Start Action (down): CONV_2D_0_4
eliminate_transposes_actions - CONV_2D_0_4 delete transpose out[0]
eliminate_transposes_actions - CONV_2D_0_4 set hint out[0] to ['c', 'h', 'w']
eliminate_transposes_actions - MUL_0_5 set transpose in[0] to (1, 2, 0)
eliminate_transposes_actions - End Action (down): MUL_0_5
eliminate_transposes_actions - Start Action (down): CONV_2D_0_19
eliminate_transposes_actions - CONV_2D_0_19 delete transpose out[0]
eliminate_transposes_actions - CONV_2D_0_19 set hint out[0] to ['c', 'h', 'w']
eliminate_transposes_actions - MUL_0_20 set transpose in[0] to (1, 2, 0)
eliminate_transposes_actions - End Action (down): MUL_0_20
eliminate_transposes_actions - Start Action (down): CONV_2D_0_22
eliminate_transposes_actions - CONV_2D_0_22 delete transpose out[0]
eliminate_transposes_actions - CONV_2D_0_22 set hint out[0] to ['c', 'h', 'w']
eliminate_transposes_actions - MUL_0_23 set transpose in[0] to (1, 2, 0)
eliminate_transposes_actions - End Action (down): MUL_0_23
eliminate_transposes_actions - Start Action (down): CONV_2D_0_25
eliminate_transposes_actions - CONV_2D_0_25 delete transpose out[0]
eliminate_transposes_actions - CONV_2D_0_25 set hint out[0] to ['c', 'h', 'w']
eliminate_transposes_actions - End Action (down): output_1
nngraph - update graph dimensions
set_aliases - looking for aliased edges
nngraph - calculate liveness
eliminate_transposes - search for transposes
eliminate_transposes - ++ Starting up from MUL_0_2[0]
eliminate_transposes - looking up at CONV_2D_0_1_activation[0] transpose (1, 2, 0)
eliminate_transposes - looking up at CONV_2D_0_1[0] transpose (1, 2, 0)
eliminate_transposes - rejected CONV_2D_0_1 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_1_activation
eliminate_transposes - Rejected up edges from MUL_0_2
eliminate_transposes - ++ Starting up from CONV_2D_0_4[0]
eliminate_transposes - looking up at ADD_0_3[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_3 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_4
eliminate_transposes - ++ Starting up from MUL_0_5[0]
eliminate_transposes - looking up at CONV_2D_0_4_activation[0] transpose (1, 2, 0)
eliminate_transposes - looking up at CONV_2D_0_4[0] transpose (1, 2, 0)
eliminate_transposes - rejected CONV_2D_0_4 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_4_activation
eliminate_transposes - Rejected up edges from MUL_0_5
eliminate_transposes - ++ Starting up from MAX_POOL_2D_0_7[0]
eliminate_transposes - looking up at ADD_0_6[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_6 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from MAX_POOL_2D_0_7
eliminate_transposes - ++ Starting down from MAX_POOL_2D_0_7[0]
eliminate_transposes - looking down at RESHAPE_0_8[0] transpose [1, 2, 0]
eliminate_transposes - reshape RESHAPE_0_8 trans: old [1, 2, 0] new None shape: old 8x4x32 new 1024
eliminate_transposes - rejected RESHAPE_0_8 - transpose out - does not reverse
eliminate_transposes - Rejected down edges from MAX_POOL_2D_0_7[0]
eliminate_transposes - ++ Starting up from CONV_2D_0_19[0]
eliminate_transposes - looking up at RESIZE_NEAREST_NEIGHBOR_0_18[0] transpose [2, 0, 1]
eliminate_transposes - rejected RESIZE_NEAREST_NEIGHBOR_0_18 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_19
eliminate_transposes - ++ Starting up from MUL_0_20[0]
eliminate_transposes - looking up at CONV_2D_0_19_activation[0] transpose (1, 2, 0)
eliminate_transposes - looking up at CONV_2D_0_19[0] transpose (1, 2, 0)
eliminate_transposes - rejected CONV_2D_0_19 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_19_activation
eliminate_transposes - Rejected up edges from MUL_0_20
eliminate_transposes - ++ Starting up from CONV_2D_0_22[0]
eliminate_transposes - looking up at ADD_0_21[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_21 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_22
eliminate_transposes - ++ Starting up from MUL_0_23[0]
eliminate_transposes - looking up at CONV_2D_0_22_activation[0] transpose (1, 2, 0)
eliminate_transposes - looking up at CONV_2D_0_22[0] transpose (1, 2, 0)
eliminate_transposes - rejected CONV_2D_0_22 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_22_activation
eliminate_transposes - Rejected up edges from MUL_0_23
eliminate_transposes - ++ Starting up from CONV_2D_0_25[0]
eliminate_transposes - looking up at ADD_0_24[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_24 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_25
eliminate_transposes - Equalizing multi input transposes on node MUL_0_2
eliminate_transposes - Trying candidate (1, 2, 0)
eliminate_transposes - (1, 2, 0) rev [2, 0, 1] trans_in None
eliminate_transposes - ++ Starting up from MUL_0_2[1]
eliminate_transposes - looking up at CAE_batch_normalization_8_FusedBatchNormV31_97d0[0] transpose [2, 0, 1]
eliminate_transposes - accepted CAE_batch_normalization_8_FusedBatchNormV31_97d0 - constant input - transpose constant [2, 0, 1]
eliminate_transposes - ++ Found results for MUL_0_2[1]
eliminate_transposes - Found equalization to (1, 2, 0) for MUL_0_2
eliminate_transposes - Equalizing multi input transposes on node MUL_0_5
eliminate_transposes - Trying candidate (1, 2, 0)
eliminate_transposes - (1, 2, 0) rev [2, 0, 1] trans_in None
eliminate_transposes - ++ Starting up from MUL_0_5[1]
eliminate_transposes - looking up at CAE_batch_normalization_9_FusedBatchNormV31_e4e9[0] transpose [2, 0, 1]
eliminate_transposes - accepted CAE_batch_normalization_9_FusedBatchNormV31_e4e9 - constant input - transpose constant [2, 0, 1]
eliminate_transposes - ++ Found results for MUL_0_5[1]
eliminate_transposes - Found equalization to (1, 2, 0) for MUL_0_5
eliminate_transposes - Equalizing multi input transposes on node MUL_0_20
eliminate_transposes - Trying candidate (1, 2, 0)
eliminate_transposes - (1, 2, 0) rev [2, 0, 1] trans_in None
eliminate_transposes - ++ Starting up from MUL_0_20[1]
eliminate_transposes - looking up at CAE_batch_normalization_10_FusedBatchNormV31_2125[0] transpose [2, 0, 1]
eliminate_transposes - accepted CAE_batch_normalization_10_FusedBatchNormV31_2125 - constant input - transpose constant [2, 0, 1]
eliminate_transposes - ++ Found results for MUL_0_20[1]
eliminate_transposes - Found equalization to (1, 2, 0) for MUL_0_20
eliminate_transposes - Equalizing multi input transposes on node MUL_0_23
eliminate_transposes - Trying candidate (1, 2, 0)
eliminate_transposes - (1, 2, 0) rev [2, 0, 1] trans_in None
eliminate_transposes - ++ Starting up from MUL_0_23[1]
eliminate_transposes - looking up at CAE_batch_normalization_11_FusedBatchNormV31_0593[0] transpose [2, 0, 1]
eliminate_transposes - accepted CAE_batch_normalization_11_FusedBatchNormV31_0593 - constant input - transpose constant [2, 0, 1]
eliminate_transposes - ++ Found results for MUL_0_23[1]
eliminate_transposes - Found equalization to (1, 2, 0) for MUL_0_23
eliminate_transposes - eliminate transposes
eliminate_transposes_actions - Start Action (up): MUL_0_2
eliminate_transposes_actions - MUL_0_2 set transpose in[1] to (1, 2, 0)
eliminate_transposes_actions - moving transpose from in to out
eliminate_transposes_actions - CAE_batch_normalization_8_FusedBatchNormV31_97d0 reorder constant input to (2, 0, 1)
eliminate_transposes_actions - End Action (up): CAE_batch_normalization_8_FusedBatchNormV31_97d0
eliminate_transposes_actions - Start Action (up): MUL_0_5
eliminate_transposes_actions - MUL_0_5 set transpose in[1] to (1, 2, 0)
eliminate_transposes_actions - moving transpose from in to out
eliminate_transposes_actions - CAE_batch_normalization_9_FusedBatchNormV31_e4e9 reorder constant input to (2, 0, 1)
eliminate_transposes_actions - End Action (up): CAE_batch_normalization_9_FusedBatchNormV31_e4e9
eliminate_transposes_actions - Start Action (up): MUL_0_20
eliminate_transposes_actions - MUL_0_20 set transpose in[1] to (1, 2, 0)
eliminate_transposes_actions - moving transpose from in to out
eliminate_transposes_actions - CAE_batch_normalization_10_FusedBatchNormV31_2125 reorder constant input to (2, 0, 1)
eliminate_transposes_actions - End Action (up): CAE_batch_normalization_10_FusedBatchNormV31_2125
eliminate_transposes_actions - Start Action (up): MUL_0_23
eliminate_transposes_actions - MUL_0_23 set transpose in[1] to (1, 2, 0)
eliminate_transposes_actions - moving transpose from in to out
eliminate_transposes_actions - CAE_batch_normalization_11_FusedBatchNormV31_0593 reorder constant input to (2, 0, 1)
eliminate_transposes_actions - End Action (up): CAE_batch_normalization_11_FusedBatchNormV31_0593
nngraph - update graph dimensions
set_aliases - looking for aliased edges
nngraph - calculate liveness
eliminate_transposes - search for transposes
eliminate_transposes - ++ Starting down from MUL_0_2[0]
eliminate_transposes - looking down at ADD_0_3[0] transpose (1, 2, 0)
eliminate_transposes - accepted ADD_0_3 - propagated transpose to broadcasted linear op
eliminate_transposes - ++ Found results for MUL_0_2[0]
eliminate_transposes - ++ Starting up from CONV_2D_0_4[0]
eliminate_transposes - looking up at ADD_0_3[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_3 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_4
eliminate_transposes - ++ Starting down from MUL_0_5[0]
eliminate_transposes - looking down at ADD_0_6[0] transpose (1, 2, 0)
eliminate_transposes - accepted ADD_0_6 - propagated transpose to broadcasted linear op
eliminate_transposes - ++ Found results for MUL_0_5[0]
eliminate_transposes - ++ Starting up from MAX_POOL_2D_0_7[0]
eliminate_transposes - looking up at ADD_0_6[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_6 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from MAX_POOL_2D_0_7
eliminate_transposes - ++ Starting down from MAX_POOL_2D_0_7[0]
eliminate_transposes - looking down at RESHAPE_0_8[0] transpose [1, 2, 0]
eliminate_transposes - reshape RESHAPE_0_8 trans: old [1, 2, 0] new None shape: old 8x4x32 new 1024
eliminate_transposes - rejected RESHAPE_0_8 - transpose out - does not reverse
eliminate_transposes - Rejected down edges from MAX_POOL_2D_0_7[0]
eliminate_transposes - ++ Starting up from CONV_2D_0_19[0]
eliminate_transposes - looking up at RESIZE_NEAREST_NEIGHBOR_0_18[0] transpose [2, 0, 1]
eliminate_transposes - rejected RESIZE_NEAREST_NEIGHBOR_0_18 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_19
eliminate_transposes - ++ Starting down from MUL_0_20[0]
eliminate_transposes - looking down at ADD_0_21[0] transpose (1, 2, 0)
eliminate_transposes - accepted ADD_0_21 - propagated transpose to broadcasted linear op
eliminate_transposes - ++ Found results for MUL_0_20[0]
eliminate_transposes - ++ Starting up from CONV_2D_0_22[0]
eliminate_transposes - looking up at ADD_0_21[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_21 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_22
eliminate_transposes - ++ Starting down from MUL_0_23[0]
eliminate_transposes - looking down at ADD_0_24[0] transpose (1, 2, 0)
eliminate_transposes - accepted ADD_0_24 - propagated transpose to broadcasted linear op
eliminate_transposes - ++ Found results for MUL_0_23[0]
eliminate_transposes - ++ Starting up from CONV_2D_0_25[0]
eliminate_transposes - looking up at ADD_0_24[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_24 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_25
eliminate_transposes - eliminate transposes
eliminate_transposes_actions - Start Action (down): MUL_0_2
eliminate_transposes_actions - MUL_0_2 delete transpose out[0]
eliminate_transposes_actions - ADD_0_3 set transpose in[0] to (1, 2, 0)
eliminate_transposes_actions - End Action (down): ADD_0_3
eliminate_transposes_actions - Start Action (down): MUL_0_5
eliminate_transposes_actions - MUL_0_5 delete transpose out[0]
eliminate_transposes_actions - ADD_0_6 set transpose in[0] to (1, 2, 0)
eliminate_transposes_actions - End Action (down): ADD_0_6
eliminate_transposes_actions - Start Action (down): MUL_0_20
eliminate_transposes_actions - MUL_0_20 delete transpose out[0]
eliminate_transposes_actions - ADD_0_21 set transpose in[0] to (1, 2, 0)
eliminate_transposes_actions - End Action (down): ADD_0_21
eliminate_transposes_actions - Start Action (down): MUL_0_23
eliminate_transposes_actions - MUL_0_23 delete transpose out[0]
eliminate_transposes_actions - ADD_0_24 set transpose in[0] to (1, 2, 0)
eliminate_transposes_actions - End Action (down): ADD_0_24
nngraph - update graph dimensions
set_aliases - looking for aliased edges
nngraph - calculate liveness
eliminate_transposes - search for transposes
eliminate_transposes - ++ Starting up from ADD_0_3[0]
eliminate_transposes - looking up at MUL_0_2[0] transpose (1, 2, 0)
eliminate_transposes - rejected MUL_0_2 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from ADD_0_3
eliminate_transposes - ++ Starting up from CONV_2D_0_4[0]
eliminate_transposes - looking up at ADD_0_3[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_3 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_4
eliminate_transposes - ++ Starting up from ADD_0_6[0]
eliminate_transposes - looking up at MUL_0_5[0] transpose (1, 2, 0)
eliminate_transposes - rejected MUL_0_5 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from ADD_0_6
eliminate_transposes - ++ Starting up from MAX_POOL_2D_0_7[0]
eliminate_transposes - looking up at ADD_0_6[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_6 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from MAX_POOL_2D_0_7
eliminate_transposes - ++ Starting down from MAX_POOL_2D_0_7[0]
eliminate_transposes - looking down at RESHAPE_0_8[0] transpose [1, 2, 0]
eliminate_transposes - reshape RESHAPE_0_8 trans: old [1, 2, 0] new None shape: old 8x4x32 new 1024
eliminate_transposes - rejected RESHAPE_0_8 - transpose out - does not reverse
eliminate_transposes - Rejected down edges from MAX_POOL_2D_0_7[0]
eliminate_transposes - ++ Starting up from CONV_2D_0_19[0]
eliminate_transposes - looking up at RESIZE_NEAREST_NEIGHBOR_0_18[0] transpose [2, 0, 1]
eliminate_transposes - rejected RESIZE_NEAREST_NEIGHBOR_0_18 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_19
eliminate_transposes - ++ Starting up from ADD_0_21[0]
eliminate_transposes - looking up at MUL_0_20[0] transpose (1, 2, 0)
eliminate_transposes - rejected MUL_0_20 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from ADD_0_21
eliminate_transposes - ++ Starting up from CONV_2D_0_22[0]
eliminate_transposes - looking up at ADD_0_21[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_21 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_22
eliminate_transposes - ++ Starting up from ADD_0_24[0]
eliminate_transposes - looking up at MUL_0_23[0] transpose (1, 2, 0)
eliminate_transposes - rejected MUL_0_23 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from ADD_0_24
eliminate_transposes - ++ Starting up from CONV_2D_0_25[0]
eliminate_transposes - looking up at ADD_0_24[0] transpose [2, 0, 1]
eliminate_transposes - rejected ADD_0_24 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_25
eliminate_transposes - Equalizing multi input transposes on node ADD_0_3
eliminate_transposes - Trying candidate (1, 2, 0)
eliminate_transposes - (1, 2, 0) rev [2, 0, 1] trans_in None
eliminate_transposes - ++ Starting up from ADD_0_3[1]
eliminate_transposes - looking up at CAE_batch_normalization_8_FusedBatchNormV3_8991[0] transpose [2, 0, 1]
eliminate_transposes - accepted CAE_batch_normalization_8_FusedBatchNormV3_8991 - constant input - transpose constant [2, 0, 1]
eliminate_transposes - ++ Found results for ADD_0_3[1]
eliminate_transposes - Found equalization to (1, 2, 0) for ADD_0_3
eliminate_transposes - Equalizing multi input transposes on node ADD_0_6
eliminate_transposes - Trying candidate (1, 2, 0)
eliminate_transposes - (1, 2, 0) rev [2, 0, 1] trans_in None
eliminate_transposes - ++ Starting up from ADD_0_6[1]
eliminate_transposes - looking up at CAE_batch_normalization_9_FusedBatchNormV3_0ad9[0] transpose [2, 0, 1]
eliminate_transposes - accepted CAE_batch_normalization_9_FusedBatchNormV3_0ad9 - constant input - transpose constant [2, 0, 1]
eliminate_transposes - ++ Found results for ADD_0_6[1]
eliminate_transposes - Found equalization to (1, 2, 0) for ADD_0_6
eliminate_transposes - Equalizing multi input transposes on node ADD_0_21
eliminate_transposes - Trying candidate (1, 2, 0)
eliminate_transposes - (1, 2, 0) rev [2, 0, 1] trans_in None
eliminate_transposes - ++ Starting up from ADD_0_21[1]
eliminate_transposes - looking up at CAE_batch_normalization_10_FusedBatchNormV3_ef1b[0] transpose [2, 0, 1]
eliminate_transposes - accepted CAE_batch_normalization_10_FusedBatchNormV3_ef1b - constant input - transpose constant [2, 0, 1]
eliminate_transposes - ++ Found results for ADD_0_21[1]
eliminate_transposes - Found equalization to (1, 2, 0) for ADD_0_21
eliminate_transposes - Equalizing multi input transposes on node ADD_0_24
eliminate_transposes - Trying candidate (1, 2, 0)
eliminate_transposes - (1, 2, 0) rev [2, 0, 1] trans_in None
eliminate_transposes - ++ Starting up from ADD_0_24[1]
eliminate_transposes - looking up at CAE_batch_normalization_11_FusedBatchNormV3_538b[0] transpose [2, 0, 1]
eliminate_transposes - accepted CAE_batch_normalization_11_FusedBatchNormV3_538b - constant input - transpose constant [2, 0, 1]
eliminate_transposes - ++ Found results for ADD_0_24[1]
eliminate_transposes - Found equalization to (1, 2, 0) for ADD_0_24
eliminate_transposes - eliminate transposes
eliminate_transposes_actions - Start Action (up): ADD_0_3
eliminate_transposes_actions - ADD_0_3 set transpose in[1] to (1, 2, 0)
eliminate_transposes_actions - moving transpose from in to out
eliminate_transposes_actions - CAE_batch_normalization_8_FusedBatchNormV3_8991 reorder constant input to (2, 0, 1)
eliminate_transposes_actions - End Action (up): CAE_batch_normalization_8_FusedBatchNormV3_8991
eliminate_transposes_actions - Start Action (up): ADD_0_6
eliminate_transposes_actions - ADD_0_6 set transpose in[1] to (1, 2, 0)
eliminate_transposes_actions - moving transpose from in to out
eliminate_transposes_actions - CAE_batch_normalization_9_FusedBatchNormV3_0ad9 reorder constant input to (2, 0, 1)
eliminate_transposes_actions - End Action (up): CAE_batch_normalization_9_FusedBatchNormV3_0ad9
eliminate_transposes_actions - Start Action (up): ADD_0_21
eliminate_transposes_actions - ADD_0_21 set transpose in[1] to (1, 2, 0)
eliminate_transposes_actions - moving transpose from in to out
eliminate_transposes_actions - CAE_batch_normalization_10_FusedBatchNormV3_ef1b reorder constant input to (2, 0, 1)
eliminate_transposes_actions - End Action (up): CAE_batch_normalization_10_FusedBatchNormV3_ef1b
eliminate_transposes_actions - Start Action (up): ADD_0_24
eliminate_transposes_actions - ADD_0_24 set transpose in[1] to (1, 2, 0)
eliminate_transposes_actions - moving transpose from in to out
eliminate_transposes_actions - CAE_batch_normalization_11_FusedBatchNormV3_538b reorder constant input to (2, 0, 1)
eliminate_transposes_actions - End Action (up): CAE_batch_normalization_11_FusedBatchNormV3_538b
nngraph - update graph dimensions
set_aliases - looking for aliased edges
nngraph - calculate liveness
eliminate_transposes - search for transposes
eliminate_transposes - ++ Starting down from ADD_0_3[0]
eliminate_transposes - looking down at CONV_2D_0_4[0] transpose (1, 2, 0)
eliminate_transposes - accepted CONV_2D_0_4 - transpose in (1)
eliminate_transposes - ++ Found results for ADD_0_3[0]
eliminate_transposes - ++ Starting down from ADD_0_6[0]
eliminate_transposes - looking down at MAX_POOL_2D_0_7[0] transpose (1, 2, 0)
eliminate_transposes - accepted MAX_POOL_2D_0_7 - transpose in (1)
eliminate_transposes - ++ Found results for ADD_0_6[0]
eliminate_transposes - ++ Starting down from MAX_POOL_2D_0_7[0]
eliminate_transposes - looking down at RESHAPE_0_8[0] transpose [1, 2, 0]
eliminate_transposes - reshape RESHAPE_0_8 trans: old [1, 2, 0] new None shape: old 8x4x32 new 1024
eliminate_transposes - rejected RESHAPE_0_8 - transpose out - does not reverse
eliminate_transposes - Rejected down edges from MAX_POOL_2D_0_7[0]
eliminate_transposes - ++ Starting up from CONV_2D_0_19[0]
eliminate_transposes - looking up at RESIZE_NEAREST_NEIGHBOR_0_18[0] transpose [2, 0, 1]
eliminate_transposes - rejected RESIZE_NEAREST_NEIGHBOR_0_18 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_19
eliminate_transposes - ++ Starting down from ADD_0_21[0]
eliminate_transposes - looking down at CONV_2D_0_22[0] transpose (1, 2, 0)
eliminate_transposes - accepted CONV_2D_0_22 - transpose in (1)
eliminate_transposes - ++ Found results for ADD_0_21[0]
eliminate_transposes - ++ Starting down from ADD_0_24[0]
eliminate_transposes - looking down at CONV_2D_0_25[0] transpose (1, 2, 0)
eliminate_transposes - accepted CONV_2D_0_25 - transpose in (1)
eliminate_transposes - ++ Found results for ADD_0_24[0]
eliminate_transposes - eliminate transposes
eliminate_transposes_actions - Start Action (down): ADD_0_3
eliminate_transposes_actions - ADD_0_3 delete transpose out[0]
eliminate_transposes_actions - CONV_2D_0_4 set hint in[0] to ['c', 'h', 'w']
eliminate_transposes_actions - CONV_2D_0_4 delete transpose in[0]
eliminate_transposes_actions - End Action (down): CONV_2D_0_4
eliminate_transposes_actions - Start Action (down): ADD_0_6
eliminate_transposes_actions - ADD_0_6 delete transpose out[0]
eliminate_transposes_actions - MAX_POOL_2D_0_7 set hint in[0] to ['c', 'h', 'w']
eliminate_transposes_actions - MAX_POOL_2D_0_7 delete transpose in[0]
eliminate_transposes_actions - End Action (down): MAX_POOL_2D_0_7
eliminate_transposes_actions - Start Action (down): ADD_0_21
eliminate_transposes_actions - ADD_0_21 delete transpose out[0]
eliminate_transposes_actions - CONV_2D_0_22 set hint in[0] to ['c', 'h', 'w']
eliminate_transposes_actions - CONV_2D_0_22 delete transpose in[0]
eliminate_transposes_actions - End Action (down): CONV_2D_0_22
eliminate_transposes_actions - Start Action (down): ADD_0_24
eliminate_transposes_actions - ADD_0_24 delete transpose out[0]
eliminate_transposes_actions - CONV_2D_0_25 set hint in[0] to ['c', 'h', 'w']
eliminate_transposes_actions - CONV_2D_0_25 delete transpose in[0]
eliminate_transposes_actions - End Action (down): CONV_2D_0_25
nngraph - update graph dimensions
set_aliases - looking for aliased edges
nngraph - calculate liveness
eliminate_transposes - search for transposes
eliminate_transposes - ++ Starting down from MAX_POOL_2D_0_7[0]
eliminate_transposes - looking down at RESHAPE_0_8[0] transpose [1, 2, 0]
eliminate_transposes - reshape RESHAPE_0_8 trans: old [1, 2, 0] new None shape: old 8x4x32 new 1024
eliminate_transposes - rejected RESHAPE_0_8 - transpose out - does not reverse
eliminate_transposes - Rejected down edges from MAX_POOL_2D_0_7[0]
eliminate_transposes - ++ Starting up from CONV_2D_0_19[0]
eliminate_transposes - looking up at RESIZE_NEAREST_NEIGHBOR_0_18[0] transpose [2, 0, 1]
eliminate_transposes - rejected RESIZE_NEAREST_NEIGHBOR_0_18 - transpose out - cannot pass
eliminate_transposes - Rejected up edges from CONV_2D_0_19
eliminate_transposes - no transposes to eliminate found
nngraph - update graph dimensions
set_aliases - looking for aliased edges
nngraph - calculate liveness
eliminate_transposes - no further transpose sequences found
set_aliases - looking for aliased edges
nngraph - adjusted order
nngraph - update graph dimensions
set_aliases - looking for aliased edges
nngraph - calculate liveness
(NNT tan_q32.tflite 0) fusions --scale8
matcher - fusions - start remove_relus
Traceback (most recent call last):
  File "/home/manucar/.local/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/manucar/.local/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/manucar/.local/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/manucar/gap_sdk/tools/nntool/interpreter/commands/fusions.py", line 65, in do_fusions
    fusion.match(self.G)
  File "/home/manucar/gap_sdk/tools/nntool/graph/matches/matcher.py", line 128, in match
    has_modified_graph = match_instance.match(G, set_identity=False)
  File "/home/manucar/gap_sdk/tools/nntool/graph/matches/remove_relus.py", line 104, in match
    qrec = G.quantization[NodeId(node)]
  File "/home/manucar/gap_sdk/tools/nntool/quantization/quantization_set.py", line 44, in __getitem__
    item = self.qset[key]
KeyError: <utils.node_id.NodeId object at 0x7f75b1f36550>
EXCEPTION of type 'KeyError' occurred with message: '<utils.node_id.NodeId object at 0x7f75b1f36550>'
(NNT tan_q32.tflite 0) 

This is my quantized .tflite model cae_2d_tanh_quant.zip.

I have also tried generating :

• a full 8 bit integer quantized model (also input and output layer) cae_2d_tanh_quant_int8.zip

• eliminating all the batch normalization layers nobatch_tanh_quant_int8.zip

but i get the same error as above.

Thank you again,

Manuel

Filter larger than input …

reshape (Reshape)            (None, **4, 2**, 32)          0        
x = Conv2D(filters=32, kernel_size=**(2, 4)**, padding='same', activation="relu")(x)

The batch norm position looks good. Must be a converter problem. Try getting rid of tf.lite.OpsSet.SELECT_TF_OPS in the TOCO options. You should check the resulting TFLITE with something like netron and make sure you get rid of those mul/adds. The mul we might be able to fuse through the relu but the add can’t be.

This was a missing feature for constant folding on our side. It will be supported in the next release. Should go out sometime next week. There are still strange things in your graph:

Here you have a filter that is larger than the input. Perhaps a bug on your side. Definitely not optimal.

Also are you placing batch normalization correctly. It is getting converted as mul / adds. It’s probably in the wrong place in your original graph.