react-native-vision-camera: 🐛 : V3, useSharedValue is not working with V3

What’s happening?

hi Marc, I’m currently working on developing a vision-camera-pose-detection project with MLKit. I initially wrote the native code for iOS, and it connected perfectly. However, I encountered a problem when attempting to draw the overlay for the skeleton based on the keypoints.

In general, I used useSharedValue from Reanimated V3 to pass a value. when I use useSharedValue I receive an error message like this: Reading from '_value' directly is only possible on the UI runtime.

I have already wrapped my JS function into a worklet, but I still get an error.

If I pass the value into setState it works, but it doesn’t behave as expected. there is a delay, as shown in this video. this is my friend who tested it

https://github.com/mrousavy/react-native-vision-camera/assets/49748249/1517267b-fcd5-4125-9715-97a561803644

"react-native-vision-camera": "^3.0.0", "react-native-worklets-core": "^0.2.0", "react-native-reanimated": "^3.4.2"

Reproduceable Code

PoseDetection.ts

import {VisionCameraProxy, Frame} from 'react-native-vision-camera';
import {IPoseDetection} from '../utils/frameProcessor/PoseDetection';

const plugin = VisionCameraProxy.getFrameProcessorPlugin('poseDetection');

export function poseDetection(frame: Frame): IPoseDetection {
  'worklet';

  if (plugin == null) {
    throw new Error('Failed to load Frame Processor Plugin "poseDetection"!');
  }

  return plugin.call(frame) as unknown as IPoseDetection;
}

Main Code:

...
// POSE

  const pose = useSharedValue<IPoseDetection>(defaultPose);

  const convertToWorklet = (poseObject: IPoseDetection, frame: Frame) => {
    const xFactor = dimensions.width / frame.width;
    const yFactor = dimensions.height / frame.height;

    const poseCopy: IPoseDetection = {
      leftShoulder: {x: 0, y: 0},
      rightShoulder: {x: 0, y: 0},
      leftElbow: {x: 0, y: 0},
      rightElbow: {x: 0, y: 0},
      leftWrist: {x: 0, y: 0},
      rightWrist: {x: 0, y: 0},
      leftHip: {x: 0, y: 0},
      rightHip: {x: 0, y: 0},
      leftKnee: {x: 0, y: 0},
      rightKnee: {x: 0, y: 0},
      leftAnkle: {x: 0, y: 0},
      rightAnkle: {x: 0, y: 0},
    };

    Object.keys(poseObject).forEach(v => {
      const key = v as keyof IPoseDetection;
      poseCopy[key] = {
        x: poseObject[key].x * xFactor,
        y: poseObject[key].y * yFactor,
      };
    });

    pose.value = poseCopy;
  };

  const setToWorklet = Worklets.createRunInJsFn(convertToWorklet);

  const frameProcessor = useFrameProcessor(
    frame => {
      'worklet';
      const poseObject = poseDetection(frame);
      console.log('pose data', poseObject);
      setToWorklet(poseObject, frame);
    },
    [pose],
  );

Relevant log output

Error: Reading from `_value` directly is only possible on the UI runtime, js engine: hermes

Camera Device

No response

Device

iPhone 11 ( 15.5 )

VisionCamera Version

3.0.0

Can you reproduce this issue in the VisionCamera Example app?

• I can reproduce the issue in the VisionCamera Example app.

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project’s Code of Conduct
• I searched for similar issues in this repository and found none.