tensorflow: Internal compiler error

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: -
TensorFlow installed from (source or binary): Docker image latest-gpu-py3
TensorFlow version (use command below): 1.14.0
Python version: 3.6
Bazel version (if compiling from source): -
GCC/Compiler version (if compiling from source): -
CUDA/cuDNN version: 10.1
GPU model and memory: RTX 2080 Ti / 12 GB

I have created a fully-quantized tf lite model from a saved model. But trying to compile it with the edgetpu_compiler, I get an error:

user@ubuntu:~/tf/tensorflow1_14$ edgetpu_compiler saved_converted_linearmodel_tpu_1.14.0.tflite 
Edge TPU Compiler version 2.0.258810407
INFO: Initialized TensorFlow Lite runtime.

Internal compiler error. Aborting!

Error message is unfortunately not very helpful. The non-compiled version is loadable and produces the correct results.

I have attached the model that I try to compile, as well as its visualization (via visualize.py).

litemodel.tar.gz

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 6
Comments: 20 (1 by maintainers)

Most upvoted comments

I’ll add that I get this error when I try to compile an object detection tflite model produced by Google Cloud AutoML. Also using Edge TPU Compiler version 2.0.258810407

jbrownkramer on Sep 6, 2019

Figured out a solution that sounds stupid. I moved the folder ‘models’ with .tflite file to ‘/home/username/edgetpu’, and then the compiler works with the same compile code provided on the official website. This ‘edgetpu’ folder was created through a beginner object detection retrain example using dataset of American bulldog and Abyssinian provided on the official website.

My setup: custom dataset, mobilenet_v1 or mobilenet_v2 downloaded from coral website, coral accelerator.

On this same example code, I initially received this error after compiling with edgetpu_compiler output_tflite_graph.tflite:

Edge TPU Compiler version 2.0.291256449

Model compiled successfully in 276 ms.

Input model: output_tflite_graph.tflite
Input size: 5.34MiB
Output model: output_tflite_graph_edgetpu.tflite
Output size: 5.75MiB
On-chip memory available for caching model parameters: 7.62MiB
On-chip memory used for caching model parameters: 5.66MiB
Off-chip memory used for streaming uncached model parameters: 0.00B
Number of Edge TPU subgraphs: 1
Total number of operations: 64
Operation log: output_tflite_graph_edgetpu.log

Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.
Number of operations that will run on Edge TPU: 63
Number of operations that will run on CPU: 1
See the operation log file for individual operation details.
Error opening file for writing: output_tflite_graph_edgetpu.tflite

Internal compiler error. Aborting!

But was able to get around it after I ran with sudo, which gives the following output:

Edge TPU Compiler version 2.0.291256449

Model compiled successfully in 341 ms.

Input model: output_tflite_graph.tflite
Input size: 5.34MiB
Output model: output_tflite_graph_edgetpu.tflite
Output size: 5.75MiB
On-chip memory available for caching model parameters: 7.62MiB
On-chip memory used for caching model parameters: 5.66MiB
Off-chip memory used for streaming uncached model parameters: 0.00B
Number of Edge TPU subgraphs: 1
Total number of operations: 64
Operation log: output_tflite_graph_edgetpu.log

Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.
Number of operations that will run on Edge TPU: 63
Number of operations that will run on CPU: 1
See the operation log file for individual operation details.

Note: I didn’t have to move the files around as mentioned in the previous post.

jdcast on Feb 9, 2020