edgetpu: How to overcome `More than one subgraph is not supported`
Inference on Edge TPU is very slow if the operations are not mapped to TPU. One of the main reasons for not mapping on TPU is given to be More than one subgraph is not supported. For example, I am trying to convert a U-Net type architecture to TFLite. I noticed that the operations on the encoder are mapped on TPU but the ones on decoder are not.
converter = tf.compat.v1.lite.TFLiteConverter.from_keras_model_file("model.h5")
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8 # or tf.int8
converter.inference_output_type = tf.int8 # or tf.int8
converter.quantized_input_stats = {'serving_default_input' : (0., 1.),
'input' : (0., 1.),
'input_0' : (0., 1.)}
# converter.allow_custom_ops = True
converter.experimental_new_converter = False
tflite_model = converter.convert()
with tf.io.gfile.GFile('resnet18.tflite', 'wb') as f:
f.write(tflite_model)
compiling the above converted model gives
Edge TPU Compiler version 15.0.340273435
Model compiled successfully in 2160 ms.
Input model: resnet18.tflite
Input size: 23.66MiB
Output model: resnet18_edgetpu.tflite
Output size: 23.57MiB
On-chip memory used for caching model parameters: 6.92MiB
On-chip memory remaining for caching model parameters: 3.50KiB
Off-chip memory used for streaming uncached model parameters: 13.46MiB
Number of Edge TPU subgraphs: 1
Total number of operations: 126
Operation log: resnet18_edgetpu.log
Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.
Number of operations that will run on Edge TPU: 90
Number of operations that will run on CPU: 36
Operator Count Status
RESIZE_BILINEAR 5 Operation version not supported
ADD 16 Mapped to Edge TPU
MAX_POOL_2D 1 Mapped to Edge TPU
CONCATENATION 4 More than one subgraph is not supported
PAD 11 More than one subgraph is not supported
PAD 34 Mapped to Edge TPU
LOGISTIC 1 More than one subgraph is not supported
QUANTIZE 4 More than one subgraph is not supported
QUANTIZE 3 Mapped to Edge TPU
CONV_2D 11 More than one subgraph is not supported
CONV_2D 36 Mapped to Edge TPU
Why are some Conv2D mapped on TPU while others are not? What does More than one subgraph is not supported mean? How to avoid this to make most of the ops map on TPU?
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 21 (1 by maintainers)
@hjonnala Update: Hi Glenn, I retrained my model, and I changed the
imgszfrom 640 to 160. With an image size of 160, it seems everything is ok; I got the result same as you. Do you have any idea why with the image size of 640, I couldn’t get that?@manoj7410 I have also tried the 320x320 model and that model gets compiled by the edgetpu. I am even getting fast inference with the 320x320 model. But I am trying to compare performance across different models and so I was hoping I could also compile the 640x640 model using the edgetpu compiler.
Do you know why the 640x640 model is not getting compiled by using the -a flag on the edgetpu compiler? Or any suggestions on how I can debug it? I can compile the 640x640 model using the edgetpu without the -a flag however the inference is very slow.
@Namburger I tried using the -a flag however I get an error saying:
Internal compiler error. Aborting!I am trying to compile the ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8 from the tensorflow object detection model zoo. I have trained this model on my own custom dataset and I have used the export_tflite_graph_tf2.py script to export my model. Without the -a flag, this is the compiler output that I get: