Get ThreeJS coordinates of mesh intersection directly below mouse click

Questions
, ,
1

I am building a 3D data annotation tool that will allow users to draw 3D bounding boxes around objects in things like medical scans, that I am loading in as a mesh. I would like to allow for some assisted annotations, specifically I want to be able to draw a 3D box around whatever part of the mesh the user clicks on.

I have a point-cloud representation of the background scan saved, and I can easily provide the bounding box coordinates of any object inside the scan, provided I know the location in ThreeJS coordinates where the click is happening. The issue is that I have orbit controls enabled, so after rotating the background scan the mouse coordinates X and Y do not correspond to the local X and Y of the scan inside ThreeJS. I have tried raycasting but have been unsuccessful because of the coordinates not matching.

This is my rayCaster code (which may be wrong). I was unable to get the casting to work using the camera I am actually viewing the mesh with, so I had to create another camera to use for casting.

    e.preventDefault();
mouse_position.x = (e.clientX / renderer.domElement.clientWidth) * 2 - 1;
mouse_position.y = -(e.clientY / renderer.domElement.clientHeight) * 2 + 1;

console.log(mouse_position)

const ray_camera = new THREE.PerspectiveCamera(50, container.offsetWidth / container.offsetHeight, 1, 35000)
rayCaster.setFromCamera(mouse_position, ray_camera);
const mesh = scene.getObjectByName("current_background")
console.log(mesh)
const intersects = rayCaster.intersectObjects([mesh], true);
console.log(intersects.length)
if (intersects.length > 0) {
    console.log(intersects[0].point)
}

This is the camera I’m using to view the scene.
new THREE.PerspectiveCamera(50, aspect, 0.01, 30000);

The issue is that when I inspect the mesh object that I want to annotate, I see that the position is {x: 0, y: 0, z: 0} so it is centered at the origin. The geometry bounding box is {"max": {x: 68.31328582763672, y: 50.31889343261719, z: 46.79656982421875}, "min": {x: -68.31328582763672, y: --50.31889343261719, z: -46.79656982421875}. When I use rayCaster and log the point of intersection, it is always a point very similar to this regardless of where I click in the scene: {x: 0.3289284805381883, y: 2.5495817904626703, z: -2.22214220339741}.

My intuition tells me that I can use the camera’s position to determine where in ThreeJS space the click is happening. I love this community and you have all helped me more than you know. Any help you can offer would be greatly appreciated.

2

Please always use the camera for raycasting you are actually rendering the scene with.

Besides, if you are not using a fullscreen canvas, you might have to adjust the computation of your mouse coordiantes, see Custom canvas size with orbitcontrols and raycaster - #2 by Mugen87.

Apart from that, I’m afraid I do not fully understand your issue. Are raycasting against a point cloud or mesh? Besides, the intersection point (intersects[0].point) is always in world space. Object3D.position which is defined in local space. Depending on whether the 3D object has a parent or not, you can’t compare both values.

3

I have adjusted the mouse coordinates to account for the canvas, I believe.

Apart from that, I’m afraid I do not fully understand your issue. Are raycasting against a point cloud or mesh? Besides, the intersection point ( intersects[0].point ) is always in world space. Object3D.position which is defined in local space. Depending on whether the 3D object has a parent or not, you can’t compare both values.

I am raycasting against a mesh. I have a point cloud representation saved server side, and using this point cloud representation I am able to cluster points using a variable distance between the points parameter in order to determine what piece of the overall point cloud I’m dealing with, and then compute the bounding box for that piece. I can then use those bounding box coordinates calculated on the server side to draw a cube in ThreeJS.

My issue is I need to know the position in local space that I have clicked so that I can feed those coordinates to my algorithms on the server side. Is there any way to convert the intersection point from world space coordinates to the mesh local space coordinates. The mesh object is listed as a “child” of the THREE.Scene

After reading your comment, I have tried

let p = intersects[0].point
let local_p = mesh.worldToLocal(p)

However I get the same vector for both p and local_p

4

Using Object3D.worldToLocal() is the right approach. However, you have to ensure that matrixWorld is up-tp-date. Please try:

const p = intersects[0].point
mesh.updateWorldMatrix(true, false)
const local_p = mesh.worldToLocal(p)
1 Like
5

That was it. It is working now. Thanks so much!

1 Like