react-native-vision-camera: 🐛 libc++abi: terminating with uncaught exception of type facebook::jsi::JSError: Exception in HostFunction: Javascript worklet error

What were you trying to do?

Using custom frame processor.

Created this new issue since the old one was closed.

Package details -

"react-native": "0.69.0",
"react-native-reanimated": "2.10.0",
"react-native-vision-camera": "2.14.1",

Reproduceable Code

QFrameProcessor.js


// eslint-disable-next-line import/no-duplicates
import 'react-native-reanimated';
import { Camera, useCameraDevices, useFrameProcessor } from 'react-native-vision-camera';
// eslint-disable-next-line import/no-duplicates
import React from 'react';

const processFrame = ( frame ) => {
  'worklet';

  // eslint-disable-next-line no-undef
  return __processFrame( frame );
};

const QFrameProcessor = ( props ) => {
  const devices = useCameraDevices( 'wide-angle-camera' );
  const device = devices.back;

  const frameProcessor = useFrameProcessor( ( frame ) => {
    'worklet';

    const data = processFrame( frame );
    console.log( 'data', data );
  }, [] );

  if ( device === undefined || !frameProcessor ) {
    return null;
  }

  return (
    <Camera
      isActive
      style={props.style}
      device={device}
      frameProcessor={frameProcessor}
      frameProcessorFps={1} />
  );
};

export default React.memo( QFrameProcessor );

QFrameProcessorPlugin.m

#import <VisionCamera/FrameProcessorPlugin.h>
#import <VisionCamera/Frame.h>

@interface QFrameProcessorPlugin : NSObject
@end

@implementation QFrameProcessorPlugin

static inline id processFrame(Frame* frame, NSArray* args) {
  CMSampleBufferRef buffer = frame.buffer;
  UIImageOrientation orientation = frame.orientation;
  // code goes here
  return @[];
}

VISION_EXPORT_FRAME_PROCESSOR(processFrame)

@end

Ideally the frame processor should have started sending me an empty array to the console.log

What happened instead?

The following error was thrown in Xcode console and the app crashed.

Relevant log output

2022-10-27 09:36:48.545198+0530 quizizz_game_v3_app[11942:1108041] [native] VisionCamera.didSetProps(_:): Updating 10 prop(s)...
2022-10-27 09:36:48.545253+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureCaptureSession(): Configuring Session...
2022-10-27 09:36:48.545262+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureCaptureSession(): Initializing Camera with device com.apple.avfoundation.avcapturedevice.built-in_video:6...
2022-10-27 09:36:48.545273+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureCaptureSession(): Adding Video input...
2022-10-27 09:36:48.545556+0530 quizizz_game_v3_app[11942:1108312] FrameProcessorBindings: Setting new frame processor...
2022-10-27 09:36:48.545578+0530 quizizz_game_v3_app[11942:1108312] FrameProcessorBindings: Adapting Shareable value from function (conversion to worklet)...
2022-10-27 09:36:48.545708+0530 quizizz_game_v3_app[11942:1108312] FrameProcessorBindings: Successfully created worklet!
2022-10-27 09:36:48.545754+0530 quizizz_game_v3_app[11942:1108390] FrameProcessorBindings: Converting worklet to Objective-C callback...
2022-10-27 09:36:48.546358+0530 quizizz_game_v3_app[11942:1108390] FrameProcessorBindings: Frame processor set!
2022-10-27 09:36:48.589822+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureCaptureSession(): Adding Video Data output...
2022-10-27 09:36:48.590355+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.invokeOnInitialized(): Camera initialized!
2022-10-27 09:36:48.590399+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureCaptureSession(): Session successfully configured!
2022-10-27 09:36:48.590872+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureFormat(): Configuring Format...
2022-10-27 09:36:48.590877+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureDevice(): Configuring Device...
2022-10-27 09:36:48.590905+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureDevice(): Device successfully configured!
2022-10-27 09:36:48.591010+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.didSetProps(_:): Starting Session...
2022-10-27 09:36:48.997963+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.didSetProps(_:): Started Session!
libc++abi: terminating with uncaught exception of type facebook::jsi::JSError: Exception in HostFunction: Javascript worklet error

Error: Javascript worklet error
    at _f (native)

Error: Exception in HostFunction: Javascript worklet error

Error: Javascript worklet error
    at _f (native)
    at _f (native)
terminating with uncaught exception of type facebook::jsi::JSError: Exception in HostFunction: Javascript worklet error

Error: Javascript worklet error
    at _f (native)

Error: Exception in HostFunction: Javascript worklet error

Error: Javascript worklet error
    at _f (native)
    at _f (native)

Device

iPhone 14 ( iOS 16.1 )

VisionCamera Version

2.14.1

Additional information

I am using Expo
I have read the Troubleshooting Guide
I agree to follow this project’s Code of Conduct
I searched for similar issues in this repository and found none.

The patch to fix Xcode 14 was already applied, this occurs even after adding that patch. Reference to the patch - https://github.com/mrousavy/react-native-vision-camera/issues/1244#issuecomment-1261364351

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 23 (6 by maintainers)

Most upvoted comments

Finally figured out the issue. After reverting back from https://github.com/mrousavy/react-native-vision-camera/pull/1308 we need to add @interface objc_name (FrameProcessorPlugin) <FrameProcessorPluginBase> previously it was just @interface objc_name (FrameProcessorPlugin) .

#define VISION_EXPORT_SWIFT_FRAME_PROCESSOR(name, objc_name)                        \
objc_name : NSObject<FrameProcessorPluginBase>                                      \
@end                                                                                \
                                                                                    \
@interface objc_name (FrameProcessorPlugin) <FrameProcessorPluginBase>              \
@end                                                                                \
@implementation objc_name (FrameProcessorPlugin)                                    \
                                                                                    \
__attribute__((constructor)) static void VISION_CONCAT(initialize_, objc_name)()    \
{                                                                                   \
  [FrameProcessorPluginRegistry addFrameProcessorPlugin:@"__" @ #name callback:^id(Frame* frame, NSArray<id>* args) {    \
    return [objc_name callback:frame withArgs:args];                               \
  }];                                                                               \
}

@mrousavy we need to make the above change for all the frame processors based on swift to work.

+15

QSuraj on Sep 22, 2023

I hope this fix will soon be released 😃

medonja on Sep 26, 2023

when is this being released in the react-native-vision camera? this is not added to the 2.16.2 please add a commit with this patch.

DarkMatter513 on Oct 12, 2023

Finally figured out the issue. After reverting back from #1308 we need to add @interface objc_name (FrameProcessorPlugin) <FrameProcessorPluginBase> previously it was just @interface objc_name (FrameProcessorPlugin) .

#define VISION_EXPORT_SWIFT_FRAME_PROCESSOR(name, objc_name)                        \
objc_name : NSObject<FrameProcessorPluginBase>                                      \
@end                                                                                \
                                                                                    \
@interface objc_name (FrameProcessorPlugin) <FrameProcessorPluginBase>              \
@end                                                                                \
@implementation objc_name (FrameProcessorPlugin)                                    \
                                                                                    \
__attribute__((constructor)) static void VISION_CONCAT(initialize_, objc_name)()    \
{                                                                                   \
  [FrameProcessorPluginRegistry addFrameProcessorPlugin:@"__" @ #name callback:^id(Frame* frame, NSArray<id>* args) {    \
    return [objc_name callback:frame withArgs:args];                               \
  }];                                                                               \
}

@mrousavy we need to make the above change for all the frame processors based on swift to work.

This fix solves the crash during app launch, thanks!

degeloper34 on Sep 22, 2023

Finally figured out the issue. After reverting back from #1308 we need to add @interface objc_name (FrameProcessorPlugin) <FrameProcessorPluginBase> previously it was just @interface objc_name (FrameProcessorPlugin) .

@mrousavy we need to make the above change for all the frame processors based on swift to work.

I can confirm that this fixes the problem!

joren-vos-aca on Sep 22, 2023

@QSuraj does it work if you revert the change in #1308 for you on Xcode 15?

reverted, the initial app crash avoided, but later the app caught an error when the camera calling the frame processor (with https://github.com/rodgomesc/vision-camera-code-scanner , @0.2.0):

Property '__scanCodes' doesn't exist

ReferenceError: Property '__scanCodes' doesn't exist
at scanBarcodes ([PROJECT PATH]/node_modules/vision-camera-code-scanner/src/index.ts (300:7):1:52)
at scanBarcodes (native)
at _f ([PROJECT PATH]/src/components/QRCodeScanner.js [REDACTED])
at _f (native)
at _f ([PROJECT PATH]/node_modules/react-native-vision-camera/src/hooks/useFrameProcessor.ts (28:21):1:425)
at _f (native)

note: camera still running but the error appears whenever it calls the frame processor

vincicat on Sep 20, 2023