react-native-vision-camera: ๐Ÿ› Camera is not aligned properly when using `useSkiaFrameProcessor`

Whatโ€™s happening?

๐Ÿ› [^4.0.0-beta.16] - Camera is not aligned properly when using useSkiaFrameProcessor but itโ€™s work with Screenshot_20240420_133231

Reproduceable Code

return (
    <SafeAreaView style={styles.container}>
      {!hasPermission && <Text>No Camera Permission.</Text>}
      {hasPermission && device != null && (
        <Camera
          style={StyleSheet.absoluteFill}
          device={device}
          isActive={true}
          photo={true}
          video={true}
          format={format}
          onError={(err)=> { console.log("err **",err)}}
          fps={fps}
          enableFpsGraph={true}
          frameProcessor={frameProcessor}  
          videoHdr={format.supportsVideoHdr}
          photoHdr={format.supportsPhotoHdr}
          enableZoomGesture={false}
        />
      )}
    </SafeAreaView>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
  },
});

Relevant log output

none

Camera Device

none

Device

vivo y21

VisionCamera Version

[^4.0.0-beta.16]

Can you reproduce this issue in the VisionCamera Example app?

I didnโ€™t try (โš ๏ธ your issue might get ignored & closed if you donโ€™t try this)

Additional information

About this issue

  • Original URL
  • State: open
  • Created 2 months ago
  • Comments: 18 (14 by maintainers)

Most upvoted comments

I will never understand why people ignore the fact that I need log output. I will write a bot in the future to auto-close issues that donโ€™t have valid log output.

then you will have zero issues in vc ๐Ÿ˜…

i think that i found a fix for this, should send a pr some time tomorrow

then itโ€™s probably the sensor to buffer transform matrix.

Try playing around with the code from useSkiaFrameProcessor: https://github.com/mrousavy/react-native-vision-camera/blob/3ff02cc6b4fc34154ec6fb5e650e074e34408e47/package/src/skia/useSkiaFrameProcessor.ts#L165-L175

Maybe weโ€™re doing something wrong.

Also, the ImageProxy (which is what the native Frame type holds) contains a getSensorToBufferTransformMatrix() method - maybe we can use this Matrix to transform the Frame safely to view dimensions? cc @rodgomesc if you wanna look into this.