react-native-vision-camera: πŸ› Crashes on getPlanes() for YUV format (V3 RC5)

What were you trying to do?

Updating from 3.0.0-rc.4 to 3.0.0-rc.5

Reproduceable Code

const frameProcessor = useFrameProcessor(frame => {
    'worklet';
    examplePlugin(frame);
  }, []);

<Camera
  frameProcessor={frameProcessor}
  pixelFormat={'yuv'}

//Took original example plugin from code and added 2 lines to ExampleFrameProcessorPlugin.callback
Image image = frame.getImage();
Log.d("ExamplePlugin", Integer.toString(image.getPlanes().length));

What happened instead?

The app crashes on image.getPlanes(). The same when I use frame.toByteBuffer() instead of image.getPlanes().

Relevant log output

ExamplePlugin           com.test D  1280 x 960 Image with format #35. Logging 5 parameters:
com.test                com.test E  lockYCbCrImpl:341 failed with 'Error3::BAD_VALUE'
AndroidMediaUtils       com.test W  lockAsyncYCbCr failed with error 3 (format = 0x23)
com.test                com.test E  lockImpl:295 failed with 'Error3::BAD_VALUE'
Gralloc3                com.test W  lock(0x748046aa69e0, ...) failed: 3
AndroidMediaUtils       com.test E  Lock buffer failed!
AndroidMediaUtils       com.test E  lockImageFromBuffer: lock graphic buffer failed
System.err              com.test W  java.lang.RuntimeException: lock buffer failed for format 0x23
System.err              com.test W  	at android.media.ImageReader$SurfaceImage.nativeCreatePlanes(Native Method)
System.err              com.test W  	at android.media.ImageReader$SurfaceImage.getPlanes(ImageReader.java:1289)
System.err              com.test W  	at com.test.ExampleFrameProcessorPlugin.callback(ExampleFrameProcessorPlugin.java:52)
System.err              com.test W  	at com.mrousavy.camera.frameprocessor.FrameProcessor.call(Native Method)
System.err              com.test W  	at com.mrousavy.camera.CameraSession.onVideoFrameCaptured(CameraSession.kt:227)
System.err              com.test W  	at com.mrousavy.camera.utils.outputs.CameraOutputs._init_$lambda$4(CameraOutputs.kt:130)
System.err              com.test W  	at com.mrousavy.camera.utils.outputs.CameraOutputs.$r8$lambda$Rkf6S8U9W_w2j3C0-q1bBmy5uA0(Unknown Source:0)
System.err              com.test W  	at com.mrousavy.camera.utils.outputs.CameraOutputs$$ExternalSyntheticLambda1.onImageAvailable(Unknown Source:2)
System.err              com.test W  	at android.media.ImageReader$1.run(ImageReader.java:947)
System.err              com.test W  	at android.os.Handler.handleCallback(Handler.java:958)
System.err              com.test W  	at android.os.Handler.dispatchMessage(Handler.java:99)
System.err              com.test W  	at android.os.Looper.loopOnce(Looper.java:205)
System.err              com.test W  	at android.os.Looper.loop(Looper.java:294)
System.err              com.test W  	at android.os.HandlerThread.run(HandlerThread.java:67)

Device

Android

VisionCamera Version

3.0.0-rc.5

Additional information

About this issue

  • Original URL
  • State: closed
  • Created 10 months ago
  • Comments: 29 (12 by maintainers)

Most upvoted comments

Tried version 3.0.0. Having new error.

ImageReader_JNI         com.test E  Producer output buffer format: 0x1, ImageReader configured format: 0x23
AndroidRuntime          com.test E  FATAL EXCEPTION: main
	Process: com.test, PID: 18726
	java.lang.UnsupportedOperationException: The producer output buffer format 0x1 doesn't match the ImageReader's configured buffer format 0x23.
		at android.media.ImageReader.nativeImageSetup(Native Method)
		at android.media.ImageReader.acquireNextSurfaceImage(ImageReader.java:597)
		at android.media.ImageReader.acquireNextImage(ImageReader.java:652)
		at android.media.ImageReader.acquireLatestImage(ImageReader.java:541)
		at com.mrousavy.camera.core.VideoPipeline.getImageReader$lambda$2(VideoPipeline.kt:99)
		at com.mrousavy.camera.core.VideoPipeline.$r8$lambda$RBt5oiVcrkwyTbFDf1xzJJfIdPg(Unknown Source:0)
		at com.mrousavy.camera.core.VideoPipeline$$ExternalSyntheticLambda0.onImageAvailable(Unknown Source:2)
		at android.media.ImageReader$1.run(ImageReader.java:947)
		at android.os.Handler.handleCallback(Handler.java:958)
		at android.os.Handler.dispatchMessage(Handler.java:99)
		at android.os.Looper.loopOnce(Looper.java:205)
		at android.os.Looper.loop(Looper.java:294)
		at android.app.ActivityThread.main(ActivityThread.java:8176)
		at java.lang.reflect.Method.invoke(Native Method)
		at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:552)
		at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:971)

I have the same Problem with β€œ3.0.0-rc.8” Maybe issue with Android Simulator ?

Closing as this is a stale issue - this might have been fixed with the full rewrite in VisionCamera V3 (πŸ₯³) - if not, please create a new issue.

If your issue has been fixed, consider sponsoring me on GitHub to say thanks πŸ’–

JFYI; We now use the GPU HardwareBuffer instead of getPlanes πŸ˜ƒ

Yes, native and rgb both work, but YUV is mandatory for the frame processors I am using.

Hey - yes the pixelFormat has to be specified - i don’t know which pixel format you want. Per default, it is the most efficient platform native format, which is PRIVATE on Android, and YUV420 on iOS.

I’ll add some docs on that for FPs πŸ˜ƒ

Also, rewriting the video pipeline now so the getPlanes and pixel formats will be greatly improved.