Raycast Highlight with InstancedMesh

I’m using InstancedMesh for my app. I would like to highlight an instance when picked using raycast. I see that a raycast intersect returns the instanceId . There is an example - webgl_instancing_raycast.html where this is used to interactively trigger a rotation for the picked sphere. How would I apply this approach to highlight/unhighlight an instance. Note: I am using the approach of examples/webgl_instancing_modified.html that adds an instanceColor attribute to a pre-existing shader. It all works fine. I just need to add highlighting.

By highlighting do you mean for example changing the color of one instance from blue to yellow, or do you want to use emission mapping?

You need an additional buffer attribute for storing original colors of instances.

  var instanceColorsBase = new Float32Array(instanceColors.length);
  instanceColorsBase.set(instanceColors);
  geometry.setAttribute( 'instanceColor', new THREE.InstancedBufferAttribute( new Float32Array( instanceColors ), 3 ) );
  geometry.setAttribute( 'instanceColorBase', new THREE.BufferAttribute(new Float32Array( instanceColorsBase ), 3 ) );

instanceColorBase is the attribute you restore instanceColor from.

Also you need to remember instanceId of previous raycast, so it’s prevInstanceId.
And a color of highlighting: var highlightColor = new THREE.Color("yellow");

Then, when you check intersection:

    if (instanceId != prevInstanceId) {
      setInstanceColor(instanceId, true);
      setInstanceColor(prevInstanceId, false);
      prevInstanceId = instanceId;
    }

Where setInstanceColor() is:

function setInstanceColor(instanceId, isHighlighting){
  if (instanceId == -1) return;
  mesh.geometry.attributes.instanceColor.setXYZ(
    instanceId,
    isHighlighting ? highlightColor.r : mesh.geometry.attributes.instanceColorBase.getX(instanceId),
    isHighlighting ? highlightColor.g : mesh.geometry.attributes.instanceColorBase.getY(instanceId),
    isHighlighting ? highlightColor.b : mesh.geometry.attributes.instanceColorBase.getZ(instanceId)
  );
  mesh.geometry.attributes.instanceColor.needsUpdate = true;
}

Means, when isHighlighting is true, then we get the color data of highlightColor, otherwise, we restore it from the instanceColorBase attribute.

Made it from the scratch at midnight, so this solution can be optimized and improved :blush:

1 Like

didnt see the duplicate, posted in the other, but it can be solved without the additional buffer attrib by keeping the previous color around: https://codesandbox.io/s/r3f-instanced-colors-8fo01 but that only works if only one item can be highlighted and the original colors must be part of the state somewhere. the instanceId we get from the raytracer is an index, so it corresponds to the color array colors[id], this way we can find the original color back.

if (hovered !== previous) {
  tempColor.set(id === hovered ? 'white' : colors[id]).toArray(colorArray, id * 3)
  geometry.attributes.color.needsUpdate = true
}
2 Likes

@drcmda very cool option! :slight_smile:

1 Like

Sorry about the duplicate post. I thought I had mistakenly deleted the duplicate.

Just swap colors when I mouseover an instance.

Thanks for this. One question (I am not yet a React’er) I see you update tempColor but you never update the color at index instanceId. How is the instance’s color updating on mouseover. Note, I see you update tempObject to create to zoom/scale effect but there you do assign tempObject.matrix back to the associated instance so that makes sense to me. What am I missing regarding the color?

This is done like so: tempColor.set(id === hovered ? 'white' : colors[id]).toArray(colorArray, id * 3). temporary objects because you dont want to re-create it 60 times per second.

the last bit is the important one: .toArray(bufferArray, index * 3) it writes itself into the float32array color buffer-attribute, then needsUpdate is called. id is the current looped id, hovered is the instanceid that’s hovered.

1 Like

Ahh. Missed the trailing parenthesis following the ternary expression. Neat.

Hi all, @prisoner849,
apologies for raising this topic back again, I thought someone might help - I need to combine both the
highlight-object-on-hover with the change-color-when-clicked.

Goal: highlight object’s color on mousemove/hover and fallback to original object color when mouseout, and highlight persistently clicked objects until clicked on another object or when the intersection array is empty.

My code below works well the hovering or the clicked functions are triggered independently, but they mess up colors when combined.

// Mouse clicking interaction: user clicked an object.
if (isClick === true) {
    if (intersects.length > 0) {
        // Get the clicked or hovered object. 
        object = intersects[0].object;
        // Get the id of the clicked/hovered object.
        const instid = intersects[0].instanceId;
        // Stop this if the object id is not valid. 
        if (instid === undefined) return;
        // If the id of the selected instance differs to the previous, then highlight the object.
        if (instid !== prevInstanceIdClick) {
            // Get the color of the clicked or hovered object.
            object.getColorAt( instid, prevInstanceColorClick)
            // Apply the per-object-default color to the previous object.
            object.setColorAt( prevInstanceIdClick, prevInstanceColorClick);
            //if (c.equals(highlightColor)) return;
            // Apply the highlight color to the current object.
            object.setColorAt( instid, highlightColor);
        }
        // store the object id for future iterations.
        prevInstanceIdClick = instid;
        // This is to tell the GPU to update the object color. 
        object.instanceColor.needsUpdate = true;
    } 
    // Mouse clicking interaction: user clicked without picking any object. 
    else if (object !== undefined) {
        // Apply the per-object-default color to the previous object.
        object.setColorAt( prevInstanceIdClick, prevInstanceColorClick);
        // Reset the previous instance id to null value.
        prevInstanceIdClick = undefined;
        // This is to tell the GPU to update the object color.
        object.instanceColor.needsUpdate = true;
    }
}
if (isClick === false) {
    if (intersects.length > 0) {
        // Get the clicked or hovered object. 
        object = intersects[0].object;
        // Get the id of the clicked/hovered object.
        const instid = intersects[0].instanceId;
        // Stop this if the object id is not valid. 
        if (instid === undefined) return;
        // If the id of the selected instance differs to the previous, then highlight the object.
        if (instid !== prevInstanceIdHover && ) {
            // Get the color of the clicked or hovered object.
            object.getColorAt( instid, prevInstanceColorHover)
            // Apply the per-object-default color to the previous object.
            object.setColorAt( prevInstanceIdHover, prevInstanceColorHover);
            // Apply the highlight color to the current object.
            object.setColorAt( instid, white); 
        }
        // store the object id for future iterations.
        prevInstanceIdHover = instid;
        // This is to tell the GPU to update the object color.
        object.instanceColor.needsUpdate = true;
    } 
    // Mouse hovering interaction: cursor is not over an object.
    else if (object !== undefined) {
        // Apply the per-object-default color to the previous object.
        object.setColorAt( prevInstanceIdHover, prevInstanceColorHover);
        // Reset the previous instance id to null value.
        prevInstanceIdHover = undefined;
        // This is to tell the GPU to update the object color.
        object.instanceColor.needsUpdate = true;
    }
}


map.on('mousemove', e => {
    // e.point is passed as a simple object like {x: float, y: float}
    customLayerBufferInstances1.raycast(e.point, false);
});

map.on('click', e => {
    // e.point is passed as a simple object like {x: float, y: float}
    customLayerBufferInstances1.raycast(e.point, true);
});

Appreciate any heads-up on this. I thought to consider a callback function to restore the original color after the user interaction, but I am not sure if that’s a good and performant option.
Thanks!