Raycast to detect animation models?

I want to detect when the user in my ThreeJS is looking at one of the animation models. Currently I am using this code:

            ourRaycaster.setFromCamera(g_Mouse, g_ThreeJsCamera);
            
            g_AryObjectsLookedAt = ourRaycaster.intersectObjects(g_ThreeJsScene.children, true);

The problem is that although the raycaster will detect any collisions between the current camera line of sight and a child object in the animation models group, it will not intersect with any empty spaces in the group. Although this makes sense from a technical point of view, it doesn’t match what we do as humans when looking at an object. When we look at an object, we mentally draw a shape around the overall object and if we are looking inside that pseudo-shape we feel that we are looking at the object.

For example, if you have a big head and wide shoulders, but a very thin neck, if I focus momentarily at the space alongside your neck I still feel that I am looking at “you”.

What I need is a raycaster that when it passes through an animation model’s group of children, does approximately the same thing.

Here is a concrete example. I am using the sample robot animation model from the ThreeJS examples. It has a really big head. When I do the raycast from this distance, no part of the robot is in the intersecting objects list:

But if I get right up “in its face”, it is detected (it’s “face” is detected):

This is technically accurate but I need something a lot more forgiving so I can create proper logic for interactions between animation model and the user. How can I do this without having to iterate manually with every animation model in the virtual scene?

You can add some simple boxes / spheres to your model. Make sure they cover all the parts that you want to be able to click. Then intersect your raycaster with the boxes / spheres in order to get user clicks on the model. The boxes and spheres should have .visible set to false

Now if you can’t do that by hand, because you don’t know in advance what model you’ll be loading, or you have a lot of models, it’s a harder problem.

I think you could use three-mesh-bvh to get a certain amount of bounding boxes from the model. Then some math wizardry ( which is left as an exercise to the reader) to create more boxes in order to make sure there are no holes in this box version of the model…

If you have each part of the model as a separate mesh, it would be simpler. Just get the bounding box for each mesh. Intersect each box with all the other boxes. If a box has 0 intersections, find the closest box to it and stretch it towards that box .

1 Like

Clever idea, thanks!

I got this comment on Stack Overflow:

"Raycaster has a ray property which is the ray used to perform the intersection. Ray has methods like intersectBox and intersectSphere, which can be used to check for intersection with a mesh’s bounding box or bounding sphere, which usually take up more space than the mesh its self. You will likely need to adjust for transformations. You could also make a Box3 or Sphere to contain the entire model (if it’s made of multiple meshes) and use the same technique to detect intersections

Will those methods work or do you think the box/sphere approach is better?

I really wish I could someone’s “deep dive” blog post on raycasting or a good set of raycasting specific samples I could peruse.