react-native-vision-camera: 🐛 `useFrameProcessor` causes app to crash [Expo]

What’s happening?

[There is a similar issue, but #2465 is closed, so starting new] [I also originally reported issue on Discord channel, but now confirmed is related to just useProcessor] [I DID NOT try to replicate in your example repo, because I am using Expo and it’s not a 1:1 comparison]

Steps to replicate:

1. Within repo link below, run dev build & open app
2. You will see camera running in view finder
3. toggle the frameProcessor prop passed to camera (here)
4. App will now crash

Notes:

I am unable to locate any meaningful crash logs to my untrained eye, but it’s definitely from within the useFrameProcessor hook, since if I remove it the app does not crash.

    "react-native": "0.72.6",
    "expo": "^49.0.1",
    "react-native-vision-camera": "^3.8.2",
    "react-native-worklets-core": "^0.3.0",

I use just the hook without any logic, copied from docs, and it crashes with no output:

  const frameProcessor = useFrameProcessor(frame => {
    'worklet';
    console.log(`Frame: ${frame.width}x${frame.height} (${frame.pixelFormat})`);
  }, []);

I was able to go into the package code & debug the JS: something about createFrameProcessor code is not being triggered??

export function useFrameProcessor(frameProcessor: (frame: Frame) => void, dependencies: DependencyList): FrameProcessor {
  console.log('inside useFrameProcessor package'); // this gets run in console
  
  return useMemo(() => {
    console.log('createProcessor'); // this gets run in console
    const process = createFrameProcessor(frameProcessor, 'frame-processor'); // no logs from inside get triggered.
  
  debugger; // this does not get triggered
  return process;
  }, dependencies)

Reproduceable Code

• https://github.com/lucksp/rnvc-test

• Install latest EAS Dev Build to run the repo in development mode:

  • https://expo.dev/accounts/lucksp/projects/rnvc-test/builds/ed8e64d9-7de2-4f14-8aa1-322de7bffa95

• (Scan QR code inside link)

Code Snippet:

export function useCamera() {
  // UNCOMMENT HERE TO ENABLE & SEE APP CRASH
  let frameProcessor;
  // const frameProcessor = useFrameProcessor(frame => {
  //   'worklet';
  //   console.log(`Frame: ${frame.width}x${frame.height} (${frame.pixelFormat})`);
  // }, []);

  const cameraRef = useRef<Component<CameraProps>>(null);
  const { hasPermission, requestPermission } = useCameraPermission();
  const device = useCameraDevice('back', {
    physicalDevices: ['ultra-wide-angle-camera', 'wide-angle-camera'],
  });

  console.log(JSON.stringify(device, (k, v) => k === "formats" ? [] : v, 2))


  const neutralZoom = device?.neutralZoom ?? MINZOOM;
  const minZoom = device?.minZoom ?? MINZOOM;
  const maxZoom = Math.min(device?.maxZoom ?? MINZOOM, MAXZOOM);

  const [isCapturing, setIsCapturing] = useState(false);
  const isFocused = useIsFocused();
  const isForeground = useIsForeground();
  const isActive = isFocused && isForeground;

  const format = useCameraFormat(device, Templates.FrameProcessingRGB);

  useEffect(
    function checkPermissions() {
      // TODO - open nice permissions UI & if denied link to settings
      if (!hasPermission) {
        requestPermission();
      }
    },
    [hasPermission, requestPermission]
  );

  return {
    device,
    maxZoom,
    minZoom,
    Camera,
    isActive,
    cameraRef,
    isCapturing,
    setIsCapturing,
    format,
    frameProcessor,
  };
}

Wrap my UI in a provider:

const CameraProvider = ({ children }: {children: ReactNode;}) => {
  const camera = useCamera();

  return (
    // eslint-disable-next-line @typescript-eslint/ban-ts-comment
    // @ts-ignore
    <CameraContext.Provider value={{ ...camera }}>
      {children}
    </CameraContext.Provider>
  );
};

build UI:

const CameraView = () => {
  const { device, Camera, cameraRef, isActive, format, frameProcessor } =
    useCameraContext();
  return (
    <>
      {device ? (
        <>
          <View>
            {/* <Reanimated.View> */}
            <Camera
              // animatedProps={animatedProps}
              device={device}
              format={format}
              frameProcessor={frameProcessor}
              isActive={isActive}
              photo
              ref={cameraRef}
            />
            {/* </Reanimated.View> */}
          </View>
        </>
      ): <Text>Loading Camera or not found</Text>}
    </>
  );
};

Relevant log output

error	12:00:01.350017-0700	SpringBoard	Advisor: No handle found for currently focused PID: 6045; sceneIdentity: com.apple.frontboard.systemappservices::FBSceneManager:sceneID%3Acom.lucksp.rnvctest-default
error	12:00:01.350045-0700	SpringBoard	Advisor: No handle found for currently focused PID: 6045; sceneIdentity: com.apple.frontboard.systemappservices::FBSceneManager:sceneID%3Acom.lucksp.rnvctest-default
error	12:00:01.583403-0700	SpringBoard	Ignoring update for invalidated scene: (FBSceneManager):sceneID:com.lucksp.rnvctest-default
error	12:00:01.583424-0700	SpringBoard	Ignoring update for invalidated scene: (FBSceneManager):sceneID:com.lucksp.rnvctest-default
error	12:00:01.584615-0700	SpringBoard	Scene FBSceneManager/sceneID:com.lucksp.rnvctest-default update failed: <NSError: 0x28007eb50; domain: FBSceneErrorDomain; code: 1 ("operation-failed"); "Scene update failed."> {
    NSUnderlyingError = <NSError: 0x280012070; domain: FBWorkspace; code: 1; "Scene client is invalid.">;
}
error	12:00:01.584638-0700	SpringBoard	Scene FBSceneManager/sceneID:com.lucksp.rnvctest-default update failed: <NSError: 0x28007eb50; domain: FBSceneErrorDomain; code: 1 ("operation-failed"); "Scene update failed."> {
    NSUnderlyingError = <NSError: 0x280012070; domain: FBWorkspace; code: 1; "Scene client is invalid.">;
}
error	12:00:01.617273-0700	runningboardd	RBSStateCapture remove item called for untracked item <RBConnectionClient| 6045 name:app<com.lucksp.rnvctest(DD75145D-5214-4263-AF89-DCD614B687C6)> entitlements:<RBEntitlements| [
			
			]> inheritanceManager:<RBClientInheritanceManager| >>
error	12:00:01.617295-0700	runningboardd	RBSStateCapture remove item called for untracked item <RBConnectionClient| 6045 name:app<com.lucksp.rnvctest(DD75145D-5214-4263-AF89-DCD614B687C6)> entitlements:<RBEntitlements| [
			
			]> inheritanceManager:<RBClientInheritanceManager| >>
error	12:00:01.618284-0700	symptomsd	COSMCtrl _foregroundAppActivity incoming bundle com.lucksp.rnvctest has nil supplied UUID, finds existing EA936BA4-CD05-3451-9D53-AFDC9FEB2C0C
error	12:00:01.618345-0700	symptomsd	COSMCtrl _foregroundAppActivity incoming bundle com.lucksp.rnvctest has nil supplied UUID, finds existing EA936BA4-CD05-3451-9D53-AFDC9FEB2C0C

Camera Device

{
  "hardwareLevel": "full",
  "isMultiCam": true,
  "minZoom": 1,
  "maxZoom": 123.75,
  "hasTorch": true,
  "sensorOrientation": "landscape-right",
  "maxExposure": 8,
  "supportsLowLightBoost": false,
  "hasFlash": true,
  "minExposure": -8,
  "name": "Back Dual Wide Camera",
  "supportsRawCapture": false,
  "neutralZoom": 2,
  "supportsFocus": true,
  "physicalDevices": [
    "ultra-wide-angle-camera",
    "wide-angle-camera"
  ],
  "position": "back",
  "formats": [],
  "minFocusDistance": 2,
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:6"
}

Device

iPhone 13Pro

VisionCamera Version

^3.8.2

Can you reproduce this issue in the VisionCamera Example app?

I didn’t try (⚠️ your issue might get ignored & closed if you don’t try this) (see note at top)

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project’s Code of Conduct
• I searched for similar issues in this repository and found none.