three.js: Bug in r119 attributes?

Description of the problem

I’m not 100% sure this is a bug vs my bad code but r115 shows a yellow square (which is what I expect) and r119 shows a black square, same code

https://jsfiddle.net/greggman/42gm165d/ (r119) https://jsfiddle.net/greggman/74w5tzub/ (r115)

First let me explain my code. I’m trying to use morph targets with both positions and vertex colors. three.js doesn’t support vertex color morphing so I’m hacking those in.

Three.js, can be given a bunch of morph targets (say 100 targets), and then based on the influences of those 100 targets it will choose up to 4 of them to actually pass to the shader (I think that’s how it works).

In order for me to know which color data to pass to the shader I need to know which position data three.js decided to pass to the shader so I iterate over the morphTarget0-3 attributes and check which BufferAttribute (which data) is being used

for (i = 0; i < maxLiveTargets; ++i) {

  // check if three is using a certain position morph target
  const bufferAttribute = geometry.getAttribute(`morphTarget${i}`);

  if (bufferAttribute) {

    // figure out which position data is assigned to this morphTarget
    // assume the names are like position0, position1, position23, etc..
    const positionTargetNdx = parseInt(bufferAttribute.name.substr(8));  

    // add in the corresponding color target
    geometry.setAttribute(`morphColor${i}`, arrayOfColorBufferAttributes[positionTargetNdx]);
  }

}

I execute the above code in onBeforeRender and it turns out which morph targets will be used is not up to date until render 3 times in r115 but it does actually get there by the 3rd render.

On r119 it never gets there.

In the example there are cube morph targets (red, yellow, green, cyan). I set the influences to 0, 1, 0, 0 so I should see a yellow cube). I added some WebGL logging. On r115 I see this

first render

-[ render ]---------------------------------------
bindBuffer: 0target (position)
bindBuffer: normal
bindBuffer: uv
bindBuffer: color0 (color)
bindBuffer: 1target (position)
bindBuffer: 2target (position)
bindBuffer: 3target (position)

---[ on before render ]---
  check attribute: morphTarget0 attributeName: (N/A) influence: 0 does NOT exist
---[end on before render]---

bindBuffer: 0target (position)
vertexAttribPointer:  position 3 FLOAT false 0 0 0target (position)
bindBuffer: 1target (position)
vertexAttribPointer:  morphTarget0 3 FLOAT false 0 0 1target (position)

First we see all these bindBuffer calls. Those are the ones used when uploading the vertex data. Then you can see at onBeforeRender there was no info about which morph targets are used so no color is set. Three then sets up 2 attributes, ‘position’ to the cube 0 (red) and morphTarget0 to cube 1 (yellow). With no color set it will be black

2nd render

-[ render ]---------------------------------------

---[ on before render ]---
  check attribute: morphTarget0 attributeName: 1morph-target-position influence: 0 exists:
  set: attrib(morphColor0) to bufferAttrib(morph-color-data1)
  check attribute: morphTarget1 attributeName: (N/A) influence: 1 does NOT exist
---[end on before render]---

bindBuffer: 0target (position)
vertexAttribPointer:  position 3 FLOAT false 0 0 0target (position)
bindBuffer: 1target (position)
vertexAttribPointer:  morphTarget0 3 FLOAT false 0 0 1target (position)

This time we see on onBeforeRender that my code saw 1morph-target-position (the yellow cube’s positions) was assigned to morphTarget0 so it assigns the corresponding color morph-color-data1 to morphColor0

But then we see three.js ignores that when actually setting up th attributes even though it sets up the attributes after onBeforeRender

3rd render

-[ render ]---------------------------------------
bindBuffer: color1 (morphColor1)

---[ on before render ]---
  check attribute: morphTarget0 attributeName: 1morph-target-position influence: 0 exists:
  set: attrib(morphColor0) to bufferAttrib(morph-color-data1)
  check attribute: morphTarget1 attributeName: (N/A) influence: 1 does NOT exist
---[end on before render]---

bindBuffer: 0target (position)
vertexAttribPointer:  position 3 FLOAT false 0 0 0target (position)
bindBuffer: color1 (morphColor1)
vertexAttribPointer:  morphColor0 3 UNSIGNED_BYTE true 0 0 color1 (morphColor1)
bindBuffer: 1target (position)
vertexAttribPointer:  morphTarget0 3 FLOAT false 0 0 1target (position)

This time we see the same onBeforeRender behavior (the color data for cube1(yellow) is assigned to morphColor0) and then three.js actually uses it and we get a yellow cube.

Switching to r119

the first and second render are exactly the same as above so no need to repeat them but the 3rd render does not set up the color attribute

-[ render ]---------------------------------------
bindBuffer: color1 (morphColor1)

---[ on before render ]---
  check attribute: morphTarget0 attributeName: 1morph-target-position influence: 0 exists:
  set: attrib(morphColor0) to bufferAttrib(morph-color-data1)
  check attribute: morphTarget1 attributeName: (N/A) influence: 1 does NOT exist
---[end on before render]---

In fact no attributes are setup. My guess is the caching that was added thinks nothing needs to happen so it just binds the existing VAO and so nothing changes, the cube stays black.

Maybe I’m using three wrong so sorry if this really belongs on S.O but I have 2 question

  1. Is this bug in three?
  2. Is there a correct way to find out what morph targets are being used in the same frame (so my code isn’t 3 frames late)
Three.js version
  • Dev
  • r119
Browser
  • All of them
  • Chrome
  • Firefox
  • Internet Explorer
OS
  • All of them
  • Windows
  • macOS
  • Linux
  • Android
  • iOS
Hardware Requirements (graphics card, VR Device, …)

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 19 (11 by maintainers)

Most upvoted comments

@greggman Wouldn’t it be better for that use case to use InstancedMesh? Specially now that it supports color per instance.

I also think that VAO usage is a must for a WebGL engine. We should clearly state to not support specific use cases (like adding/removing attributes in onBeforeRender()) rather than revert VAO.

Thanks for the report. Let me investigate.

Do you want to create a new issue / feature request for adding vertex color support to morph targets?

Here’s the use case.

https://r105.threejsfundamentals.org/threejs/threejs-lots-of-objects-morphtargets-w-colors.html

Basically making the WebGL Globe but fixing the part where colors only match the first dataset.

The solution I’d prefer is to be able to ask three what it is going to do.

const morphInfo = someFunctionGetMorphInfo(someDataNeededGetComputeMorphInfo)

It doesn’t have to usable in onBeforeRender.

To be clear, I’d prefer not adding color support to three.js specifically but a solution that allows the user to add their own code and still utilize what three is already doing. That’s the point of the example. It is not “how do I morph colors”, it is “how do I add my own data to morph targets”

I can use solution takahirox posted above and just comment that this isn’t really something three wants to support and that you should really write your own custom shaders and write your own morph target system if that’s the recommended solution. I was just hoping that given the code to support morph targets is already in three.js that it was possibly to piggyback off that rather than having to go 100% custom.

Meaning… You’re hacking around an a part of the API that is hard to guarantee is not going to break. At the end of the day, what you want is to be able to do is morph between colors and not only positions, right?

Should VAO be reverted?

I don’t think (and I don’t hope) we should revert because the root issue of this problem is not from VAO as I wrote above. Adding/Removing attributes in onBeforeRender() has been problematic even since before we adopt VAO.

But yeah sorry for the bugs reported in other threads and already fixed. VAO actually improves the performance. So I prefer trying to further clean up and optimize VAO related code rather than reverting.

Conclusion

I think I have to say that adding/removing attributes to/from geometry in onBeforeRender is not recommended not only in r119 but also even in r115. I’ll explain the reason later.

Workaround

A workaround I came up with so far is picking up morph color attributes before calling renderer.render() in the same way as renderer picks up morph target attributes, like this

function onBeforeRender(mesh) {
  const geometry = mesh.geometry;

  // remove all the color attributes
  for (const {name} of colorAttributes) {
    geometry.deleteAttribute(name);
  }

  // Find the up to eight biggest influences other than 0 influence
  const influences = [];
  for (let i = 0; i < mesh.morphTargetInfluences.length; i++) {
    influences.push([i, mesh.morphTargetInfluences[i]]);
  }

  const effectiveInfluences = influences
    .sort((a, b) => Math.abs(b[1]) - Math.abs(a[1]))
    .slice(0, 8)
    .sort((a, b) => a[0] - b[0]) // comment out this line for r115
    .filter(a => !!a[1]);

  // Add corresponding color attributes
  for (let i = 0; i < effectiveInfluences.length; ++i) {
    const ndx = effectiveInfluences[i][0];
    const name = `morphColor${i}`;
    geometry.setAttribute(name, colorAttributes[ndx].attribute);
  }
}

function render() {
  onBeforeRender(scene.children[0]);
  renderer.render(scene, camera);
}

Example with r119 https://jsfiddle.net/7or8t4d1/ It looks working.

The limitation is this idea can not be used if multiple meshes share a geometry. I haven’t come up with an idea for such a case yet.

Why we shouldn’t add attributes in onBeforeRender?

The reason why we shouldn’t add/remove attributes to/from geometry in onBeforeRender is WebGL buffer management of renderer, Renderer doesn’t seem to expect attributes are updated by user in .render(). .

WebGL buffer handling and onBeforeRender() are done in this order.

  1. Create WebGL buffer and update buffer data if needed
  2. Fire Object3D.onBeforeRender()
  3. Set morph target attributes to geometry corresponding to the up to eight biggest morphTargetInfluences
  4. Switch VAO, and bind buffers if geometry is dirty
  5. Render

From this, I can explain how the problems you reported happen.

first render Then you can see at onBeforeRender there was no info about which morph targets are used so no color is set.

onBeforeRender() is fired before render sets morph attributes. Then you don’t see morph attributes in geometry.

2nd render But then we see three.js ignores that when actually setting up the attributes even though it sets up the attributes after onBeforeRender

onBeforeRender() is fired after creating WebGL buffer. Then in this frame WebGL buffers corresponding to morph color attributes are not created even if you add morph color attributes in onBeforeRender()

And morph attributes in geometry you see in onBeforeRender is one frame old.

3rd render in r115

WebGL buffer corresponding to morph color attribute is bound in the third render then you see it looks working. But note that morph attributes in geometry you see in onBeforeRender() is one frame old. So if your app for example updates morphTargetInfluences each frame, you may see unexpected morph color.

3rd render in r119

In this example, geometry isn’t changed since the previous render call. (In onBeforeRender() you remove and add the yellow morph attribute.) Then skips buffer binding. Remember that in the second render WebGL buffer corresponding to morph color attribute wasn’t created. In the third render we skips buffer bindings then morph color buffer won’t be bound. Then you keep seeing black box.

Is this a bug?

  1. Is this bug in three?

WebGL resource management in render is complex. Perhaps hard to resolve this issue “adding/removing attributes in onBeforeRender() causes problem”. So it may be reasonable to think this is a limitation of onBeforeRender() rather than this is a bug.

What do you think? @mrdoob @Mugen87