tensorflow: interpreter.invoke() of tflite model causes Aborted (core dumped) despite successful tflite conversion under tensorflow version 1.14.0

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Kubuntu 18.04
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
• TensorFlow installed from (source or binary): pip3
• TensorFlow version (use command below): 1.14.0
• Python version: 3.6.9
• Bazel version (if compiling from source):
• GCC/Compiler version (if compiling from source):
• CUDA/cuDNN version:
• GPU model and memory:

Describe the current behavior The mobilenetvlad model was successfuly converted to a tf lite model by the sample code provided from the tensorflow website. To archieve this I added the parameter input_shapes to the from_saved_model call: converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir, input_shapes={"image": [1, 640, 480, None]})

But if inference is tested with the according sample code from the tensorflow website, the program is aborted and the core dumped.

Describe the expected behavior Sucessfull inference and output of a [1, 4096] sized image descriptor.

Code to reproduce the issue

1. Download the mobilenetvlad model
2. With tensorflow 1.14.0 (tf2 will not work): Convert the saved_model to tflite, by setting an input shape (was set to None, None, None, 1]) in the model description, but should be 640x480 according to the paper.pdf:

import tensorflow as tf
saved_model_dir='hierarchical_loc/global-loc/models/mobilenetvlad_depth-0.35'
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir, input_shapes={"image": [1, 640, 480, None]}) #only change of code beside filenames
tflite_model = converter.convert()
open("converted_model_1_640_480.tflite", "wb").write(tflite_model) 

1. Run the inference with the sample code:

import numpy as np
import tensorflow as tf

print(tf.__version__)

# Load TFLite model and allocate tensors.
interpreter = tf.lite.Interpreter(model_path="converted_model_1_640_480.tflite")
interpreter.allocate_tensors()

# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
print(input_details)
print(output_details)

# Test model on random input data.
input_shape = input_details[0]['shape']
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)
print(input_data.shape)
interpreter.set_tensor(input_details[0]['index'], input_data)

interpreter.invoke()

output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)

Other info / logs

Output of the tflite conversion. [Edited: copied the wrong output, sorry]. And don’t worry about the virtual environmnet naming VirtualEnvironments/tensorflow_1_15/. I initially wanted to install tf 1.15.0, but only tf 1.14.0 worked:

  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
2020-01-22 14:00:03.226815: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-01-22 14:00:03.250753: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2194920000 Hz
2020-01-22 14:00:03.251151: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4a4df80 executing computations on platform Host. Devices:
2020-01-22 14:00:03.251185: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
WARNING:tensorflow:From /home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorflow/lite/python/convert_saved_model.py:60: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.loader.load or tf.compat.v1.saved_model.load. There will be a new function for importing SavedModels in Tensorflow 2.0.
WARNING:tensorflow:From /home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
2020-01-22 14:00:06.393038: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2020-01-22 14:00:10.226966: I tensorflow/core/grappler/devices.cc:60] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0 (Note: TensorFlow was not compiled with CUDA support)
2020-01-22 14:00:10.227122: I tensorflow/core/grappler/clusters/single_machine.cc:359] Starting new session                                                                                                                                 
2020-01-22 14:00:10.347863: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:716] Optimization results for grappler item: graph_to_optimize                                                                                          
2020-01-22 14:00:10.347910: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   function_optimizer: function_optimizer did nothing. time = 0.002ms.                                                                              
2020-01-22 14:00:10.347930: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   function_optimizer: function_optimizer did nothing. time = 0.001ms.                                                                              
WARNING:tensorflow:From /home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorflow/lite/python/util.py:238: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.                                                                                                                                                                                               
Instructions for updating:                                                                                                                                                                                                                  
Use `tf.compat.v1.graph_util.convert_variables_to_constants`                                                                                                                                                                                
WARNING:tensorflow:From /home/alex/Documents/VirtualEnvironments/tensorflow_1_15/lib/python3.6/site-packages/tensorflow/python/framework/graph_util_impl.py:270: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.                                                                                                                                                                                            
Instructions for updating:                                                                                                                                                                                                                  
Use `tf.compat.v1.graph_util.extract_sub_graph`                                                                                                                                                                                             
2020-01-22 14:00:11.150707: I tensorflow/core/grappler/devices.cc:60] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0 (Note: TensorFlow was not compiled with CUDA support)                                         
2020-01-22 14:00:11.151014: I tensorflow/core/grappler/clusters/single_machine.cc:359] Starting new session                                                                                                                                 
2020-01-22 14:00:12.806255: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:716] Optimization results for grappler item: graph_to_optimize                                                                                          
2020-01-22 14:00:12.806306: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   constant folding: Graph size after: 554 nodes (-265), 570 edges (-265), time = 1091.39502ms.                                                     
2020-01-22 14:00:12.806326: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   constant folding: Graph size after: 554 nodes (0), 570 edges (0), time = 372.614ms.      

The created model is then used within tfLiteInference.py:

import numpy as np
import tensorflow as tf

print(tf.__version__)

# Load TFLite model and allocate tensors.
interpreter = tf.lite.Interpreter(model_path="converted_model_1_640_480_test.tflite")
interpreter.allocate_tensors()

# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
print(input_details)
print(output_details)

# Test model on random input data.
input_shape = input_details[0]['shape']
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)
print(input_data.shape)
interpreter.set_tensor(input_details[0]['index'], input_data)

interpreter.invoke()

output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)

Output of gdb debugging gdb -ex r --args python3 tfLiteinference.py --> Aborted --> py-list:

Thread 1 "python3" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) py-list
 104    
 105        def AllocateTensors(self):
 106            return _tensorflow_wrap_interpreter_wrapper.InterpreterWrapper_AllocateTensors(self)
 107    
 108        def Invoke(self):
>109            return _tensorflow_wrap_interpreter_wrapper.InterpreterWrapper_Invoke(self)
 110    
 111        def InputIndices(self):
 112            return _tensorflow_wrap_interpreter_wrapper.InterpreterWrapper_InputIndices(self)
 113    
 114        def OutputIndices(self):

I hope anyone can help me with this.

Cheers, Alex