tensorflow: ruy::CpuInfo::Initialize() Null pointer dereference: SIGSEGV 0x0000000000000008

Issue type

Bug

Have you reproduced the bug with TensorFlow Nightly?

No

Source

source

TensorFlow version

tensorflow-lite:2.16.1

Custom code

No

OS platform and distribution

No response

Mobile device

Android

Python version

No response

Bazel version

6.5.0

GCC/compiler version

Clang 8.0.7

CUDA/cuDNN version

No response

GPU model and memory

No response

Current behavior?

We recently upgraded TensorFlow Lite from 2.9.0 to 2.16.1 (the latest version from MavenCentral) and received a fresh native crash issue on Firebase Crashlytic. (Possibly related to #74043). But since the official release of the shared library on MavenCentral was stripped (#72877), we can’t investigate further. Our team decided to rebuild TensorFlow Lite 2.16.1 from source, with the flag tflite_keep_symbols=true to keep the stack trace info more readable. The issue has about 30k crash events, affecting 7k users. We still, can not reproduce the crash locally.

crashlytics_report

Standalone code to reproduce the issue

We use only two main functions to infer the model, the crash happened in both cases.


org.tensorflow.lite.InterpreterApi#run
org.tensorflow.lite.Interpreter#runSignature(java.util.Map<java.lang.String,java.lang.Object>, java.util.Map<java.lang.String,java.lang.Object>, java.lang.String)


### Relevant log output

```shell
null pointer dereference: SIGSEGV  0x0000000000000008
#00 pc 0x23ddd8 libtensorflowlite_jni.so (ruy::CpuInfo::Initialize() [cpuinfo.cc:66]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#01 pc 0x23ddd4 libtensorflowlite_jni.so (ruy::CpuInfo::Initialize() [cpuinfo.cc:64]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#02 pc 0x23de6c libtensorflowlite_jni.so (ruy::CpuInfo::NeonDotprod() [cpuinfo.cc:35]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#03 pc 0x23c2c8 libtensorflowlite_jni.so (ruy::Ctx::GetRuntimeEnabledPaths() [ctx.cc:125]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#04 pc 0x23c3b4 libtensorflowlite_jni.so (ruy::Ctx::SelectPath(ruy::Path) [ctx.cc:177]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#05 pc 0xa63b4 libtensorflowlite_jni.so (void ruy::detail::CreateTrMulParamsAssumingColMajorDst<(ruy::Path)49, float, float, float, float>(ruy::Mat<float> const&, ruy::Mat<float> const&, ruy::Mat<float> const&, ruy::MulParams<float, float> const&, ruy::ChannelDimension, ruy::Ctx*, ruy::TrMulParams*) [create_trmul_params.h:423]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#06 pc 0xa626c libtensorflowlite_jni.so (void ruy::MulFrontEnd<(ruy::Path)49, float, float, float, float>(ruy::Mat<float> const&, ruy::Mat<float> const&, ruy::MulParams<float, float> const&, ruy::Ctx*, ruy::Mat<float>*) [create_trmul_params.h:472]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#07 pc 0xa5cb8 libtensorflowlite_jni.so (tflite::cpu_backend_gemm::detail::GemmImplUsingRuy<float, float, float, float, (tflite::cpu_backend_gemm::QuantizationFlavor)0>::Run(tflite::cpu_backend_gemm::MatrixParams<float> const&, float const*, tflite::cpu_backend_gemm::MatrixParams<float> const&, float const*, tflite::cpu_backend_gemm::MatrixParams<float> const&, float*, tflite::cpu_backend_gemm::GemmParams<float, float, (tflite::cpu_backend_gemm::QuantizationFlavor)0> const&, tflite::CpuBackendContext*) [ruy.h:46]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#08 pc 0x12c9bc libtensorflowlite_jni.so (tflite::optimized_ops::FullyConnected(tflite::FullyConnectedParams const&, tflite::RuntimeShape const&, float const*, tflite::RuntimeShape const&, float const*, tflite::RuntimeShape const&, float const*, tflite::RuntimeShape const&, float*, tflite::CpuBackendContext*) [optimized_ops.h:306]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#09 pc 0x12b108 libtensorflowlite_jni.so (TfLiteStatus tflite::ops::builtin::fully_connected::EvalFloat<(tflite::ops::builtin::fully_connected::KernelType)1>(TfLiteContext*, TfLiteNode*, TfLiteFullyConnectedParams*, tflite::ops::builtin::fully_connected::OpData*, TfLiteTensor const*, TfLiteTensor const*, TfLiteTensor const*, TfLiteTensor*) [fully_connected.cc:1563]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#10 pc 0x129aa0 libtensorflowlite_jni.so (TfLiteStatus tflite::ops::builtin::fully_connected::Eval<(tflite::ops::builtin::fully_connected::KernelType)1>(TfLiteContext*, TfLiteNode*) [fully_connected.cc:1605]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#11 pc 0x3403b0 libtensorflowlite_jni.so (tflite::Subgraph::InvokeImpl() [subgraph.cc:1396]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#12 pc 0x33fd98 libtensorflowlite_jni.so (tflite::Subgraph::Invoke() [subgraph.cc:1581]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#13 pc 0x333bc8 libtensorflowlite_jni.so (tflite::impl::SignatureRunner::Invoke() [signature_runner.cc:82]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#14 pc 0x28a4c libtensorflowlite_jni.so (Java_org_tensorflow_lite_NativeSignatureRunnerWrapper_nativeInvoke [nativesignaturerunner_jni.cc:268]) (BuildId: 488448e6d222d2f499e4c2978695bdc8ea291e71)
#15 pc 0x351e30 libart.so (BuildId: ddcc440d4609d2099db9d20895487a78)

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 26 (2 by maintainers)

Most upvoted comments

@sebouh00 We have recently been able to solve this issue by fully migrating to XNNPack. The issue that was preventing us from using XNNPack was a small detail on the model.

Our model didn’t have the batch size specified, causing XNNPack to fail because it doesn’t handle dynamic tensors. The error message was something in the lines of what you find in this issue: https://github.com/tensorflow/tensorflow/issues/63142

In this case, TFLite will fallback to ruy, but the only indication that this happened is an error log, so it might go unnoticed.

@prilaga Yes. We updated the ruy dependency to a version that fixes the crash, and rolled out our own TFL artifact. See here. We first forked ruy, and applied some changes to the bazel build files to make it work with TFL’s bazel version. See here.

Hi, @ninh-huynh Thanks for raising this issue. Are you aware of the migration to LiteRT? This transition is aimed at enhancing our project’s capabilities and providing improved support and focus for our users. As we believe this issue is still relevant to LiteRT we are moving your issue there. Please follow progress here: https://github.com/google-ai-edge/LiteRT/issues/36

Let us know if you have any questions. Thanks.

Could it be related to the issue that’s been addressed here?