expo: [expo-camera,expo-face-detector] ByteBuffer size and format don't match from mlkit.vision.common.InputImage with Camera2 (crash android)

Summary

Hi,

There is an issue with expo-face-detector@11.2.0 and expo-camera@12.2.0 for Android with useCamera2Api={true} props.

The expo-face-detector is trying to convert the image with a bad format (https://github.com/expo/expo/blob/main/packages/expo-face-detector/android/src/main/java/expo/modules/facedetector/ExpoFaceDetector.kt#L96).

It appears that Camera2 uses the format YUV_420_888 but expo-face-detector tries to convert the bytes with NV21, see https://developers.google.com/android/reference/com/google/mlkit/vision/common/InputImage.

Thus when trying to detect a face the app crashes with the error :

ByteBuffer size and format don't match

As a workaround you can put useCamera2Api={false}.

Managed or bare workflow? If you have `ios/` or `android/` directories in your project, the answer is bare!

bare

What platform(s) does this occur on?

Android

SDK Version (managed workflow only)

Environment

expo-env-info 1.0.3 environment info: System: OS: macOS 12.2.1 Shell: 5.8 - /bin/zsh Binaries: Node: 18.1.0 - /usr/local/bin/node Yarn: 1.22.18 - /usr/local/bin/yarn npm: 8.8.0 - /usr/local/bin/npm Watchman: 2022.03.21.00 - /usr/local/bin/watchman Managers: CocoaPods: 1.11.3 - /Users/slals/.rvm/rubies/ruby-2.7.4/bin/pod SDKs: iOS SDK: Platforms: DriverKit 21.4, iOS 15.4, macOS 12.3, tvOS 15.4, watchOS 8.5 Android SDK: API Levels: 23, 27, 28, 29, 30, 31 Build Tools: 28.0.3, 29.0.2, 30.0.2, 30.0.3, 31.0.0 System Images: android-30 | Google APIs Intel x86 Atom IDEs: Android Studio: 4.1 AI-201.8743.12.41.6953283 Xcode: 13.3.1/13E500a - /usr/bin/xcodebuild npmPackages: expo: ^45.0.4 => 45.0.4 react: 17.0.2 => 17.0.2 react-dom: ^16.8.6 => 16.14.0 react-native: 0.68.1 => 0.68.1 npmGlobalPackages: expo-cli: 5.3.0 Expo Workflow: bare

Reproducible demo

None

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 5
Comments: 28 (3 by maintainers)

Most upvoted comments

according to google ml-kit docs:

If you use the Camera2 API, capture images in ImageFormat.YUV_420_888 format. If you use the older Camera API, capture images in ImageFormat.NV21 format.

I suppose the error is related to this line:

https://github.com/expo/expo/blob/64c74995f1a142344b066de883823ad75b51cfde/packages/expo-face-detector/android/src/main/java/expo/modules/facedetector/ExpoFaceDetector.kt#L100

I think this issue should be reopened as the error still happening on these version: expo: 49.0.21 expo-camera: 13.6.0 expo-face-detector: 12.4.0

nonam4 on Jan 16, 2024

Any update on this?.I am also continuously facing this issue on android 11 even without camera2 api. I am facing this issue in load testing .it works fine for first 3/4 go

SaifullahIlyas on Dec 21, 2022

<Camera type={CameraType.front} whiteBalance={8} style={{ height: 380, width: 300 }} {…propsFaceDetector} onCameraReady={() => { setPropsFaceDetector({ onFacesDetected: onFacesDetected, faceDetectorSettings: { mode: FaceDetector.FaceDetectorMode.fast, minDetectionInterval: 500, detectLandmarks: FaceDetector.FaceDetectorLandmarks.none, runClassifications: FaceDetector.FaceDetectorClassifications.all, tracking: false, }, }); }} ref={(ref) => { setCamera(ref); }}

</Camera>

On Sun, 20 Aug 2023 at 19:42, Mudassir Rafiq @.***> wrote:

So what’s its solution?

— Reply to this email directly, view it on GitHub https://github.com/expo/expo/issues/17524#issuecomment-1685350842, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC3BOTJLFUSHXUEWXMOGVRLXWJEBXANCNFSM5WCSNXEA . You are receiving this because you commented.Message ID: @.***>

mickytroxxy on Aug 20, 2023