MediaPipeUnityPlugin: iOS crash on Pose tracking demo EXC_BAD_ACCESS
System information
- OS Platform and Distribution (e.g., Linux Ubuntu 20.04): iOS 14.8
- Target Platform: iOS 14.8
- Mobile Device (if the target is Android or iOS): iPhone XS
- MediaPipeUnityPlugin version or commit id: efc7664e3ed5d997ee618c364bd2256896a43aae
- Bazel version: 4.2.1
- GCC/G++ version:
- Android NDK version (if the target platform is Android):
- Xcode version (if the target platform is iOS): 13.1
Describe the current behavior App crash after running for while. The crash timing is random. Sometime 10 mins but sometimes can run for an hour.
Steps to reproduce the issue
- Choose pose tracking demo and run on the device for long period of time.
- Using front camera and choose 640x480 resolution
- Target the camera on a looping video that have human dance.
Full logs
See https://docs.unity3d.com/Manual/LogFiles.html
Additional Context
The above behaviour is running the demo in Async mode. Therefore, I have also tested on Sync mode. It seems the error will not occur. The demo can run for more than 2 hours in Sync mode
From the callback stack trace, I found out this is caused by the source code from mediapipe. I think Sync mode can prevent to trigger the last element in mirrors_ so the bug cannot be triggered.
mediapipe/framework/output_stream_manager.cc line 191
// mediapipe/framework/output_stream_manager.cc line 191 from https://github.com/google/mediapipe/
for (int idx = 0; idx < mirror_count; ++idx) {
const Mirror& mirror = mirrors_[idx];
if (add_packets) {
// If the stream is the last element in mirrors_, moves packets from
// output_queue_. Otherwise, copies the packets.
if (idx == mirror_count - 1) {
mirror.input_stream_handler->MovePackets(mirror.id,
packets_to_propagate);
} else {
mirror.input_stream_handler->AddPackets(mirror.id,
*packets_to_propagate);
}
}
if (set_bound) {
mirror.input_stream_handler->SetNextTimestampBound(mirror.id,
next_timestamp_bound);
}
}
Do you have any idea how to fix it? I would love to contribute in it. By the way, nice project. It helps me a lot.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 31 (14 by maintainers)
@siisee11 Thanks! I’ll wait another week or two, and if there are no reports, I’ll merge it into master.
I also tested
78b2d1aand run on IPad Mini 6 for 2 hours. The crash didn’t happened. Let’s see if there are more testing result in the future. Thanks for the fix @homulerI’ve not yet succeeded in reproducing this error, but one thing comes to mind. In asynchronous mode, each callback function returns an
IntPtr, which is a pointer toabsl::Status.https://github.com/homuler/MediaPipeUnityPlugin/blob/5222faa9a21e1c523614f8c082b92ec521c2005b/Packages/com.github.homuler.mediapipe/Runtime/Scripts/Unity/OutputStream.cs#L380
From the managed code’s point of view, this
Statusobject is subject to GC. IfStatus#Disposeis called while the underlyingabsl::Statusis still referenced by the unmanaged code, I think it can cause SIGSEGV.It depends on the implementation and how you are measuring the FPS.
In the Non-blocking mode,
GraphRunner#TryGetNextwon’t block the main thread (seeOutputStream#TryGetNextfor more details). However, if it doesn’t return the result value, we can’t tell whether it’s because MediaPipe has not finished inferencing or because the result is just empty, so the sample app waits until at least the timeout period has elapsed.https://github.com/homuler/MediaPipeUnityPlugin/blob/2497f0cae0ab60e12e1cabf5af69c27fb75fa27e/Assets/Mediapipe/Samples/Common/Scripts/ImageSourceSolution.cs#L106-L110 https://github.com/homuler/MediaPipeUnityPlugin/blob/2497f0cae0ab60e12e1cabf5af69c27fb75fa27e/Assets/Mediapipe/Samples/Scenes/Pose Tracking/PoseTrackingSolution.cs#L64-L67
So if the inference doesn’t finish in 1 frame (e.g. 16ms if running at 60fps), it will wait for a few frames. To mitigate this, you can lower the timeout period, but possibly just stopping calling
WaitUntilmay solve your problem (very similar behavior in the async mode).@grayhong Thank you for reporting the issue.
Perhaps the cause and the conditions are different, so create another issue if necessary. A recent commit (not released yet) just relates to
CreateCVPixelBufferForImageFramePacket(https://github.com/google/mediapipe/commit/cc6a2f7af65977248b2a15f471503da2832f583a#diff-bef471e93b87c12ce09a9413971a44005287f038b57d58d5e6213ef377d2d61a), but I don’t know if it’s related to this error.Can anyone test to see if https://github.com/homuler/MediaPipeUnityPlugin/commit/78b2d1a3e2c6be98249689cd54ee5d228a123498 fixes the problem?
Thanks!
It’s in a master, but it is not a fix (rather refactoring).
@laukaho @grayhong @patrick508 Can you confirm this error still occurs with the latest commit (https://github.com/homuler/MediaPipeUnityPlugin/commit/5222faa9a21e1c523614f8c082b92ec521c2005b)? While I tried running the sample app on my iPad Air (4th) for about 90 mins, I could not reproduce it.
Thank you so much @homuler! I will let you know if I find a way to reproduce this issue!
By the way, if anyone knows how to reproduce this more reliably, please let me know.
With so many things to do, it is impossible to spend an hour just for reproduction.