tensorflow: Android TFlite inconsistent performance when app is not in focus.

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Android 9.0 and 10.0
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: Pixel 3a (Android 10), tested on Android 9 as well with same bad behaviour
• TensorFlow installed from (source or binary): tflite .so built from source
• TensorFlow version (use command below): 1.13, 1.14, 1.15, 2.0, 2.1
• Bazel version (if compiling from source): various; 0.21 to 0.29

Problem: We have an audio processing application that runs in real time, including when the phone’s screen is off. Using the built shared objects (tensorflowlite.so) from r1.13 and r1.14 branches, our processing time stays consistent. Locking the phone or minimizing the app does not affect the tflite inference times. Our app also uses a foreground service to make Android give us optimal process scheduling.

However, when using newer tflite shared objects (.so) built from r1.15, r2.0, and r2.1 the inference performance drops when the app is not in focus (either screen was locked, or our app minimized). The behaviour is especially bad when using more threads, e.g: interpreter->SetNumThreads(2);

Describe the current behavior Android tflite model inference times are lower when app is not in focus with tflite versions r1.15, r2.0, r2.1. Describe the expected behavior Exactly the same inference processing performance when the app is in focus or not.

Is there something in the API of newer tflite versions (r1.15 and newer) that I could play around with to fix this? Any help is greatly appreciated, thank you!