sceneview-android: Augmented Face - Not Rendering Model and Still in "surface mode"
Hello, I was trying to understand learn and understand “sceneview” usages, I already discovered how to get Augmented face in “sceneform-android” but when I try sceneview, I encountered several problems,
- First camera settings are seems to be not working properly because some options act wrong. (For example MESH3D mode or I was able to enable Deph mode (normally I shouldn’t be able to do that in Front Camera) )
- Model size is too small
- Model moving opposite side when I move my head (If I go right, model goes left)
- Model not render properly and shows me “black material”
- I’m always seeing “Searching for surfaces…” (front camera or back camera) It should not be visible in front camera
Here my minimal code (only MainActivity)
private lateinit var binding: ActivityMainBinding
private val facesNodes = HashMap<AugmentedFace, AugmentedFaceNode>()
private var faceModel: ModelRenderable? = null
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
binding = ActivityMainBinding.inflate(layoutInflater)
setContentView(binding.root)
loadModels()
}
private fun loadModels() {
lifecycleScope.launchWhenCreated {
binding.sceneView.configureSession { arSession, config ->
val filter =
CameraConfigFilter(arSession).setFacingDirection(CameraConfig.FacingDirection.FRONT)
val cameraConfig = arSession.getSupportedCameraConfigs(filter)[0]
arSession.cameraConfig = cameraConfig
config.instantPlacementMode = Config.InstantPlacementMode.DISABLED
config.augmentedFaceMode = Config.AugmentedFaceMode.MESH3D
arSession.configure(config)
}
faceModel = loadModel(
context = this@MainActivity,
lifecycle = lifecycle,
glbFileLocation = "models/fox.glb",
)
binding.sceneView.onAugmentedFaceUpdate = {
val existingFaceNode = facesNodes[it]
when (it.trackingState) {
TrackingState.TRACKING ->
if (existingFaceNode == null) {
val faceNode = AugmentedFaceNode(binding.sceneView.lifecycle, it)
val modelInstance = faceNode.setFaceRegionsRenderable(faceModel)
modelInstance.isShadowCaster = false
modelInstance.isShadowReceiver = true
faceNode.faceRegionsRenderable = faceModel!!
binding.sceneView.addChild(faceNode)
facesNodes[it] = faceNode
}
TrackingState.STOPPED -> {
if (existingFaceNode != null) {
binding.sceneView.removeChild(existingFaceNode)
facesNodes.remove(it)
}
facesNodes.remove(it)
}
else -> {
facesNodes.remove(it)
}
}
}
}
}
activity_main.xml snippet as well
<io.github.sceneview.ar.ArSceneView
android:id="@+id/sceneView"
android:layout_width="0dp"
android:layout_height="0dp"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintLeft_toLeftOf="parent"
app:layout_constraintRight_toRightOf="parent"
app:layout_constraintTop_toTopOf="parent" />
PS : I used "fox and “face” glb files from sceneform for quick testing.
If you need additional information please let me know.
Thank you.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 24 (7 by maintainers)
Hello I find the solution, you need to put this after config change
sceneView.onArSessionResumed = { it.setCameraTextureNames(sceneView.arCameraStream.cameraTextureIds) }
I think that explanation there is a little bit wrong. The texture ID points to the OpenGL texture but it is up to ARCore to actually create this texture and fill it with camera frames. When
setTextureName
is called we connect the texture with the camera that is currently specified by the ARCoreCameraConfig
. Therefore, the texture and camera IDs don’t have to match. Moreover, when using multiple textures withsetTextureNames
each texture from the array has its own ID.It can be so that calling
setCameraConfig
resets the texture names and we need to set them again, for example, by settinghasSetTextureNames
tofalse
.For my end, I had a busy month and it is getting close to finish. I will come up with update and code change, for texture problem I had to check so I can’t answer that one atm
I made bit more progress on this and trying to stable somehow so I can share with PR and repo so everybody try it (of course try with care, it can be break 😃) )