three.js: WebGLRenderer: fails to convert CubeCamera renderTarget to PMREM CubeUV format

Do you think there’s anything blocking that?

I’ve forgot that we could use this approach now the we have migrated to half floats. I guess we can give this a try and see how the solution performs.

@WestLangley While I agree that PMREM is a much better way to do PBR environment lighting, I think it may have been a mistake to remove the less accurate but cheaper method here: https://github.com/mrdoob/three.js/pull/22234/files#diff-7fdd1b94f9a0571f64abfb392826f21068b52acb69082c9b69bbd2a4de980a24L88

For any kind of dynamic lighting situation, be it WebXR or something like the cubemap dynamic example I think it’s probably worth the accuracy loss for the speedup. In fact I was trying to understand how that example still works, and my best guess is it only works for shiny materials?

@makc Opting out is not possible. MeshPhysicalMaterial and MeshStandardMaterial only support PMREM cube maps.

The only solutions are:

1. Tell the user to use MeshPhongMaterial instead, or
2. Tell the user to compute the PMREM manually, or
3. FIx this issue.

@Mugen87 I think my proposal above would solve this better. Do you think there’s anything blocking that?

@Mugen87 In model-viewer we’re already using a cube camera to generate an HDR env map in HalfFloat; support seems to be pretty much universal now for this format. So, I believe we can now safely (I hope) switch the internal format of CubeUV from RGBE to HalfFloat, which should help performance. I believe the mip-chain alternative to PMREM deleted in #22234 will also work properly, but only with HalfFloat textures.

I propose we require envMaps to be in HalfFloat internally (converting if necessary) and then add some kind of flag to a CubeCamera render target (dynamic?) to indicate if you want the fast path with only mipmapping for an environment that updates, or the slower, more accurate PMREM for a static environment. What do you think?

Or maybe we revert #22234 until this issue is solved. I tend to favor this solution for now.

The assuption of the PR (All non-PMREM environment maps assigned to a Physical material are now converted to PMREM CubeUV format automatically by the renderer.) is actually not true because of render targets.

In AR you should see your real environment reflected in the spheres as well as the lighting but they are just lit correctly but there is no reflection.

Just a note this has also broken the environment setting in this demo: https://threejs.org/examples/?q=light#webxr_ar_lighting

I also ran into this issue recently. I’m now generating my environment in a proper, separated way, rather than using PMREM fromScene(). However, I had to apply these hacks to get the PMREM to generate: https://github.com/google/model-viewer/pull/2832/files#r730114514

@WestLangley

That is an edge case,

That is the only way to get reflections right, for example, and since it is already slow (due to rendering cubemap), slowing it down even further by default should at least have a way to opt out.