react-native-vision-camera: š Front Camera not loading in s23 ultra
Whatās happening?
@mrousavy I tried the latest v3 example code. By default front camera is loading in s23 ultra.
Then i removed everything and made it
const devices = useCameraDevices();
now back camera loads, but switching to front crashes the app.
now if i use
const device = useCameraDevice(cameraPosition, { physicalDevices: [ āultra-wide-angle-cameraā, ātelephoto-cameraā, āwide-angle-cameraā, ], });
i can see front camera. but unable to switch back to Back Camera
now if i go back to the old code and refresh the app. Then both cameras work. But if i close the app open it again, back camera will work, but front camera will fail
Reproduceable Code
CameraPage.tsx
import * as React from 'react';
import {useRef, useState, useCallback, useMemo} from 'react';
import {StyleSheet, Text, View} from 'react-native';
import {
PinchGestureHandler,
PinchGestureHandlerGestureEvent,
TapGestureHandler,
} from 'react-native-gesture-handler';
import {
CameraRuntimeError,
PhotoFile,
useCameraDevice,
//useCameraDevices,
useCameraFormat,
// useFrameProcessor,
VideoFile,
} from 'react-native-vision-camera';
import {Camera} from 'react-native-vision-camera';
import {
CONTENT_SPACING,
MAX_ZOOM_FACTOR,
SAFE_AREA_PADDING,
SCREEN_HEIGHT,
SCREEN_WIDTH,
} from './Constants';
import Reanimated, {
Extrapolate,
interpolate,
useAnimatedGestureHandler,
useAnimatedProps,
useSharedValue,
} from 'react-native-reanimated';
import {useEffect} from 'react';
import {useIsForeground} from './hooks/useIsForeground';
import {StatusBarBlurBackground} from './views/StatusBarBlurBackground';
import {CaptureButton} from './views/CaptureButton';
import {PressableOpacity} from 'react-native-pressable-opacity';
import MaterialIcon from 'react-native-vector-icons/MaterialCommunityIcons';
import IonIcon from 'react-native-vector-icons/Ionicons';
import type {Routes} from './Routes';
import type {NativeStackScreenProps} from '@react-navigation/native-stack';
import {useIsFocused} from '@react-navigation/core';
// import {examplePlugin} from './frame-processors/ExamplePlugin';
const ReanimatedCamera = Reanimated.createAnimatedComponent(Camera);
Reanimated.addWhitelistedNativeProps({
zoom: true,
});
const SCALE_FULL_ZOOM = 3;
const BUTTON_SIZE = 40;
type Props = NativeStackScreenProps<Routes, 'CameraPage'>;
export function CameraPage({navigation}: Props): React.ReactElement {
const camera = useRef<Camera>(null);
const [isCameraInitialized, setIsCameraInitialized] = useState(false);
const [hasMicrophonePermission, setHasMicrophonePermission] = useState(false);
const zoom = useSharedValue(0);
const isPressingButton = useSharedValue(false);
// check if camera page is active
const isFocussed = useIsFocused();
const isForeground = useIsForeground();
const isActive = isFocussed && isForeground;
const [cameraPosition, setCameraPosition] = useState<'front' | 'back'>(
'back',
);
const [enableHdr, setEnableHdr] = useState(false);
const [flash, setFlash] = useState<'off' | 'on'>('off');
const [enableNightMode, setEnableNightMode] = useState(false);
useEffect(() => {
getDevices();
}, []);
const getDevices = async () => {
const devicesDetected = await Camera.getAvailableCameraDevices();
// console.log('devicesDetected', devicesDetected);
return devicesDetected;
};
// const devices = useCameraDevices();
// console.log('devicesXXSD', devices);
// const device = devices[1];
// camera format settings
const device = useCameraDevice(cameraPosition, {
physicalDevices: [],
});
console.log(
'cameraPosition',
cameraPosition,
device?.name,
device?.maxZoom,
device?.formats.some(f => f.maxFps >= 60),
);
const [targetFps, setTargetFps] = useState(60);
const screenAspectRatio = SCREEN_HEIGHT / SCREEN_WIDTH;
const format = useCameraFormat(device, [
{fps: targetFps},
{videoAspectRatio: screenAspectRatio},
{videoResolution: 'max'},
{photoAspectRatio: screenAspectRatio},
{photoResolution: 'max'},
{photoHDR: enableHdr},
{videoHDR: enableHdr},
]);
const fps = Math.min(format?.maxFps ?? 1, targetFps);
const supportsFlash = device?.hasFlash ?? false;
const supportsHdr = format?.supportsPhotoHDR && format.supportsVideoHDR;
const supports60Fps = useMemo(
() => device?.formats.some(f => f.maxFps >= 60),
[device?.formats],
);
const canToggleNightMode = device?.supportsLowLightBoost ?? false;
//#region Animated Zoom
// This just maps the zoom factor to a percentage value.
// so e.g. for [min, neutr., max] values [1, 2, 128] this would result in [0, 0.0081, 1]
const minZoom = device?.minZoom ?? 1;
const maxZoom = Math.min(device?.maxZoom ?? 1, MAX_ZOOM_FACTOR);
const cameraAnimatedProps = useAnimatedProps(() => {
const z = Math.max(Math.min(zoom.value, maxZoom), minZoom);
return {
zoom: z,
};
}, [maxZoom, minZoom, zoom]);
//#endregion
//#region Callbacks
const setIsPressingButton = useCallback(
(_isPressingButton: boolean) => {
isPressingButton.value = _isPressingButton;
},
[isPressingButton],
);
// Camera callbacks
const onError = useCallback((error: CameraRuntimeError) => {
console.error(error);
}, []);
const onInitialized = useCallback(() => {
console.log('Camera initialized!');
setIsCameraInitialized(true);
}, []);
const onMediaCaptured = useCallback(
(media: PhotoFile | VideoFile, type: 'photo' | 'video') => {
console.log(`Media captured! ${JSON.stringify(media)}`);
navigation.navigate('MediaPage', {
path: media.path,
type: type,
});
},
[navigation],
);
const onFlipCameraPressed = useCallback(() => {
setCameraPosition(p => (p === 'back' ? 'front' : 'back'));
}, []);
const onFlashPressed = useCallback(() => {
setFlash(f => (f === 'off' ? 'on' : 'off'));
}, []);
//#endregion
//#region Tap Gesture
const onDoubleTap = useCallback(() => {
onFlipCameraPressed();
}, [onFlipCameraPressed]);
//#endregion
//#region Effects
const neutralZoom = device?.neutralZoom ?? 1;
useEffect(() => {
// Run everytime the neutralZoomScaled value changes. (reset zoom when device changes)
zoom.value = neutralZoom;
}, [neutralZoom, zoom]);
useEffect(() => {
Camera.getMicrophonePermissionStatus().then(status =>
setHasMicrophonePermission(status === 'granted'),
);
}, []);
//#endregion
//#region Pinch to Zoom Gesture
// The gesture handler maps the linear pinch gesture (0 - 1) to an exponential curve since a camera's zoom
// function does not appear linear to the user. (aka zoom 0.1 -> 0.2 does not look equal in difference as 0.8 -> 0.9)
const onPinchGesture = useAnimatedGestureHandler<
PinchGestureHandlerGestureEvent,
{startZoom?: number}
>({
onStart: (_, context) => {
context.startZoom = zoom.value;
},
onActive: (event, context) => {
// we're trying to map the scale gesture to a linear zoom here
const startZoom = context.startZoom ?? 0;
const scale = interpolate(
event.scale,
[1 - 1 / SCALE_FULL_ZOOM, 1, SCALE_FULL_ZOOM],
[-1, 0, 1],
Extrapolate.CLAMP,
);
zoom.value = interpolate(
scale,
[-1, 0, 1],
[minZoom, startZoom, maxZoom],
Extrapolate.CLAMP,
);
},
});
//#endregion
if (device != null && format != null) {
console.log(
`Re-rendering camera page with ${
isActive ? 'active' : 'inactive'
} camera. ` +
`Device: "${device.name}" (${format.photoWidth}x${format.photoHeight} photo / ${format.videoWidth}x${format.videoHeight} video @ ${fps}fps)`,
);
} else {
console.log('re-rendering camera page without active camera');
}
// const frameProcessor = useFrameProcessor((frame) => {
// 'worklet';
// console.log(`${frame.timestamp}: ${frame.width}x${frame.height} ${frame.pixelFormat} Frame (${frame.orientation})`);
// examplePlugin(frame);
// }, []);
return (
<View style={styles.container}>
{device != null && (
<PinchGestureHandler onGestureEvent={onPinchGesture} enabled={isActive}>
<Reanimated.View style={StyleSheet.absoluteFill}>
<TapGestureHandler onEnded={onDoubleTap} numberOfTaps={2}>
<ReanimatedCamera
ref={camera}
style={StyleSheet.absoluteFill}
device={device}
format={format}
fps={fps}
key={device.id}
hdr={enableHdr}
lowLightBoost={device.supportsLowLightBoost && enableNightMode}
isActive={isActive}
onInitialized={onInitialized}
onError={onError}
enableZoomGesture={false}
animatedProps={cameraAnimatedProps}
enableFpsGraph={true}
orientation="portrait"
photo={true}
video={true}
audio={hasMicrophonePermission}
// frameProcessor={frameProcessor}
/>
</TapGestureHandler>
</Reanimated.View>
</PinchGestureHandler>
)}
<CaptureButton
style={styles.captureButton}
camera={camera}
onMediaCaptured={onMediaCaptured}
cameraZoom={zoom}
minZoom={minZoom}
maxZoom={maxZoom}
flash={supportsFlash ? flash : 'off'}
enabled={isCameraInitialized && isActive}
setIsPressingButton={setIsPressingButton}
/>
<StatusBarBlurBackground />
<View style={styles.rightButtonRow}>
<PressableOpacity
style={styles.button}
onPress={onFlipCameraPressed}
disabledOpacity={0.4}>
<IonIcon name="camera-reverse" color="white" size={24} />
</PressableOpacity>
{supportsFlash && (
<PressableOpacity
style={styles.button}
onPress={onFlashPressed}
disabledOpacity={0.4}>
<IonIcon
name={flash === 'on' ? 'flash' : 'flash-off'}
color="white"
size={24}
/>
</PressableOpacity>
)}
{supports60Fps && (
<PressableOpacity
style={styles.button}
onPress={() => setTargetFps(t => (t === 30 ? 60 : 30))}>
<Text style={styles.text}>{`${targetFps}\nFPS`}</Text>
</PressableOpacity>
)}
{supportsHdr && (
<PressableOpacity
style={styles.button}
onPress={() => setEnableHdr(h => !h)}>
<MaterialIcon
name={enableHdr ? 'hdr' : 'hdr-off'}
color="white"
size={24}
/>
</PressableOpacity>
)}
{canToggleNightMode && (
<PressableOpacity
style={styles.button}
onPress={() => setEnableNightMode(!enableNightMode)}
disabledOpacity={0.4}>
<IonIcon
name={enableNightMode ? 'moon' : 'moon-outline'}
color="white"
size={24}
/>
</PressableOpacity>
)}
</View>
</View>
);
}
const styles = StyleSheet.create({
container: {
flex: 1,
backgroundColor: 'black',
},
captureButton: {
position: 'absolute',
alignSelf: 'center',
bottom: SAFE_AREA_PADDING.paddingBottom,
},
button: {
marginBottom: CONTENT_SPACING,
width: BUTTON_SIZE,
height: BUTTON_SIZE,
borderRadius: BUTTON_SIZE / 2,
backgroundColor: 'rgba(140, 140, 140, 0.3)',
justifyContent: 'center',
alignItems: 'center',
},
rightButtonRow: {
position: 'absolute',
right: SAFE_AREA_PADDING.paddingRight,
top: SAFE_AREA_PADDING.paddingTop,
},
text: {
color: 'white',
fontSize: 11,
fontWeight: 'bold',
textAlign: 'center',
},
});
Relevant log output
Some logs i got
{ [session/camera-has-been-disconnected: [session/camera-has-been-disconnected] The given Camera device (id: 1) has been disconnected! Error: UNKNOWN_CAMERA_DEVICE_ERROR] name: āsession/camera-has-been-disconnectedā,
`device/invalid-device: No Camera Device could be found!
`
Process: com.visioncamera, PID: 27069
java.lang.IllegalArgumentException: getCameraCharacteristics:1043: Unable to retrieve camera characteristics for unknown device 1: No such file or directory (-2)
### Camera Device
```json
[ { pixelFormats: [ 'rgb', 'native', 'yuv', 'unknown' ],
videoStabilizationModes: [ 'off', 'standard', 'cinematic', 'off', 'cinematic-extended' ],
autoFocusSystem: 'contrast-detection',
supportsPhotoHDR: true,
photoHeight: 3060,
supportsVideoHDR: true,
supportsDepthCapture: false,
maxISO: 3200,
minISO: 50,
minFps: 1,
videoWidth: 3840,
videoHeight: 2160,
photoWidth: 4080,
fieldOfView: 75.70461464712741,
maxFps: 30 },
{ pixelFormats: [ 'rgb', 'native', 'yuv', 'unknown' ],
videoStabilizationModes: [ 'off', 'standard', 'cinematic', 'off', 'cinematic-extended' ],
autoFocusSystem: 'contrast-detection',
supportsPhotoHDR: true,
photoHeight: 3000,
supportsVideoHDR: true,
supportsDepthCapture: false,
maxISO: 3200,
minISO: 50,
minFps: 1,
videoWidth: 3840,
videoHeight: 2160,
photoWidth: 4000,
fieldOfView: 75.70461464712741,
maxFps: 30 },
{ pixelFormats: [ 'rgb', 'native', 'yuv', 'unknown' ],
videoStabilizationModes: [ 'off', 'standard', 'cinematic', 'off', 'cinematic-extended' ],
autoFocusSystem: 'contrast-detection',
supportsPhotoHDR: true,
photoHeight: 2252,
supportsVideoHDR: true,
supportsDepthCapture: false,
maxISO: 3200,
minISO: 50,
minFps: 1,
videoWidth: 3840,
videoHeight: 2160,
photoWidth: 4000,
fieldOfView: 75.70461464712741,
maxFps: 30 },
{ pixelFormats: [ 'rgb', 'native', 'yuv', 'unknown' ],
videoStabilizationModes: [ 'off', 'standard', 'cinematic', 'off', 'cinematic-extended' ],
autoFocusSystem: 'contrast-detection',
supportsPhotoHDR: true,
photoHeight: 1868,
supportsVideoHDR: true,
supportsDepthCapture: false,
maxISO: 3200,
minISO: 50,
minFps: 1,
videoWidth: 3840,
videoHeight: 2160,
photoWidth: 4000,
fieldOfView: 75.70461464712741,
maxFps: 30 },
{ pixelFormats: [ 'rgb', 'native', 'yuv', 'unknown' ],
videoStabilizationModes: [ 'off', 'standard', 'cinematic', 'off', 'cinematic-extended' ],
autoFocusSystem: 'contrast-detection',
supportsPhotoHDR: true,
photoHeight: 1716,
supportsVideoHDR: true,
supportsDepthCapture: false,
maxISO: 3200,
minISO: 50,
minFps: 1,
videoWidth: 3840,
videoHeight: 2160,
photoWidth: 4000,
fieldOfView: 75.70461464712741,
maxFps: 30 },
{ pixelFormats: [ 'rgb', 'native', 'yuv', 'unknown' ],
videoStabilizationModes: [ 'off', 'standard', 'cinematic', 'off', 'cinematic-extended' ],
autoFocusSystem: 'contrast-detection',
supportsPhotoHDR: true,
photoHeight: 2160,
supportsVideoHDR: true,
supportsDepthCapture: false,
maxISO: 3200,
minISO: 50,
minFps: 1,
videoWidth: 3840,
videoHeight: 2160,
photoWidth: 3840,
fieldOfView: 75.70461464712741,
maxFps: 30 },
{ pixelFormats: [ 'rgb', 'native', 'yuv', 'unknown' ],
videoStabilizationModes: [ 'off', 'standard', 'cinematic', 'off', 'cinematic-extended' ],
autoFocusSystem: 'contrast-detection',
supportsPhotoHDR: true,
photoHeight: 2736,
supportsVideoHDR: true,
supportsDepthCapture: false,
maxISO: 3200,
minISO: 50,
minFps: 1,
videoWidth: 3840,
videoHeight: 2160,
photoWidth: 3648,
fieldOfView: 75.70461464712741,
maxFps: 30 },
{ pixelFormats: [ 'rgb', 'native', 'yuv', 'unknown' ],
videoStabilizationModes: [ 'off', 'standard', 'cinematic', 'off', 'cinematic-extended' ],
autoFocusSystem: 'contrast-detection',
supportsPhotoHDR: true,
photoHeight: 2052,
supportsVideoHDR: true,
supportsDepthCapture: false,
maxISO: 3200,
minISO: 50,
minFps: 1,
videoWidth: 3840,
videoHeight: 2160,
photoWidth: 3648,
fieldOfView: 75.70461464712741,
maxFps: [TOO BIG formatValueCalls 201 exceeded limit of 200] },
Device
S23 Ultra Android
VisionCamera Version
3.1.0
Can you reproduce this issue in the VisionCamera Example app?
Yes I can
Additional information
- I am using Expo
- I have enabled Frame Processors (react-native-worklets-core)
- I have read the Troubleshooting Guide
- I agree to follow this projectās Code of Conduct
- I searched for similar issues in this repository and found none.
About this issue
- Original URL
- State: closed
- Created 9 months ago
- Reactions: 1
- Comments: 34 (16 by maintainers)
Hey! Can anyone try with VisionCamera 3.4.1. and see if the issue still persists? Thanks!
Hey!
Quick update: I think there could be four problems here:
limited/legacydevice, and use Preview (always on),photoandvideo/frameProcessoroutputs (3 outputs) which are by default at maximum resolution. LIMITED/LEGACY devices might not support full-resolution streaming, see the Android capture session documentation on supported configuration maps. We needPRIV PREVIEW+YUV/PRIV RECORD+JPEG MAXIMUM. anything lower, needs to be explicitly handled by me, but I can easily add that no problemo. I just need the device to test it on.photo,videoandframeProcessorand check if that works. If only PREVIEW is attached, everything should work.getPreviewCaptureRequestand see what happens. Iāve heardzoomis often the cause.Okay so we need to clamp zoom