tensorflow: TFLite Mfcc op has inconsistent requirements with standard Mfcc op

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes, here’s the code to reproduce: https://gist.github.com/reuben/57bee91669a5bd2717c32cf406ca951d
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS 10.14
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
• TensorFlow installed from (source or binary): binary
• TensorFlow version (use command below): v1.13.0-rc2-5-g6612da8951 1.13.1
• Python version: 3.6.6
• Bazel version (if compiling from source): N/A
• GCC/Compiler version (if compiling from source): N/A
• CUDA/cuDNN version: N/A
• GPU model and memory: N/A
• Exact command to reproduce: curl https://gist.githubusercontent.com/reuben/57bee91669a5bd2717c32cf406ca951d/raw/6b81d28d00ff3ec73ab1bcc6a698366bbe3dcb51/test_tflite_mfcc.py | python

Describe the problem

The Mfcc op in tensorflow.contrib.framework.python.ops.audio_ops (a.k.a. contrib_audio) enforces the shape of the sample rate parameter to be rank 0. The TFLite Mfcc op enforces the sample rate parameter to be rank 1. Converting a model with an Mfcc op in it results in a TFLite model that fails in the preparation step.

Source code / logs

If you try to pass a sample rate of rank 1 to the contrib_audio op:

python3 test_tflite.py
2019-02-27 11:50:41.627337: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Traceback (most recent call last):
  File "/Users/rmorais/.local/share/virtualenvs/DeepSpeech-HmF6CP0D/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1659, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Shape must be rank 0 but is rank 1 for 'Mfcc' (op: 'Mfcc') with input shapes: [1,1,257], [1].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test_tflite.py", line 13, in <module>
    mfccs = contrib_audio.mfcc(spectrogram, [16000], dct_coefficient_count=13)
  File "/Users/rmorais/.local/share/virtualenvs/DeepSpeech-HmF6CP0D/lib/python3.6/site-packages/tensorflow/python/ops/gen_audio_ops.py", line 454, in mfcc
    dct_coefficient_count=dct_coefficient_count, name=name)
  File "/Users/rmorais/.local/share/virtualenvs/DeepSpeech-HmF6CP0D/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "/Users/rmorais/.local/share/virtualenvs/DeepSpeech-HmF6CP0D/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/Users/rmorais/.local/share/virtualenvs/DeepSpeech-HmF6CP0D/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
    op_def=op_def)
  File "/Users/rmorais/.local/share/virtualenvs/DeepSpeech-HmF6CP0D/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1823, in __init__
    control_input_ops)
  File "/Users/rmorais/.local/share/virtualenvs/DeepSpeech-HmF6CP0D/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1662, in _create_c_op
    raise ValueError(str(e))
ValueError: Shape must be rank 0 but is rank 1 for 'Mfcc' (op: 'Mfcc') with input shapes: [1,1,257], [1].

If you pass a sample rate of rank 0, and then try to convert and use the TFLite model:

python3 test_tflite.py
2019-02-27 11:50:27.082848: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
session run works
toco_from_protos /var/folders/k1/7sbxsmm52n59_fpj0v65fmk40000gn/T/tmp_vtx0s5o /var/folders/k1/7sbxsmm52n59_fpj0v65fmk40000gn/T/tmp7v9bbo3t /var/folders/k1/7sbxsmm52n59_fpj0v65fmk40000gn/T/tmpilazyufw /var/folders/k1/7sbxsmm52n59_fpj0v65fmk40000gn/T/tmp9e1513n8
Traceback (most recent call last):
  File "test_tflite.py", line 30, in <module>
    interpreter.allocate_tensors()
  File "/Users/rmorais/.local/share/virtualenvs/DeepSpeech-HmF6CP0D/lib/python3.6/site-packages/tensorflow/lite/python/interpreter.py", line 73, in allocate_tensors
    return self._interpreter.AllocateTensors()
  File "/Users/rmorais/.local/share/virtualenvs/DeepSpeech-HmF6CP0D/lib/python3.6/site-packages/tensorflow/lite/python/interpreter_wrapper/tensorflow_wrap_interpreter_wrapper.py", line 106, in AllocateTensors
    return _tensorflow_wrap_interpreter_wrapper.InterpreterWrapper_AllocateTensors(self)
RuntimeError: tensorflow/lite/kernels/mfcc.cc:75 NumDimensions(inputRate) != 1 (0 != 1)Node number 2 (Mfcc) failed to prepare.

And here’s the source of the testing script just in case:

import numpy as np
import tensorflow as tf
import sys
import tempfile

from tensorflow.contrib.framework.python.ops import audio_ops as contrib_audio


with tf.Session() as sess:
    input_ph = tf.placeholder(tf.float32, [512])
    samples = tf.reshape(input_ph, [512, 1])
    spectrogram = contrib_audio.audio_spectrogram(samples, window_size=512, stride=320, magnitude_squared=True)
    mfccs = contrib_audio.mfcc(spectrogram, 16000, dct_coefficient_count=13)

    sess.run([mfccs], feed_dict={input_ph: np.random.random([512])})
    print('session run works')

    converter = tf.lite.TFLiteConverter(sess.graph_def, input_tensors=[input_ph], output_tensors=[mfccs])
    converter.allow_custom_ops = True
    tflite_model = converter.convert()

with tempfile.NamedTemporaryFile(delete=False) as fout:
    temp_name = fout.name
    fout.write(tflite_model)
    fout.flush()

try:
    # Load TFLite model and allocate tensors.
    interpreter = tf.lite.Interpreter(model_path=temp_name)
    interpreter.allocate_tensors()
    print('tflite model prepare works')

    # Get input and output tensors.
    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()

    # Test model on random input data.
    for inp in input_details:
        input_data = np.array(np.random.random_sample(inp['shape']), dtype=np.float32)
        interpreter.set_tensor(inp['index'], input_data)

    interpreter.invoke()
    output_data = interpreter.get_tensor(output_details[0]['index'])
    print(output_data)
finally:
    try:
        os.remove(temp_name)
    except:
        pass