Intersects of raycaster.setFromCamera are strangely rotated (worldToLocal needed?)

I’m trying to get as many coordinates as possible for the surface parts of a 3D model that are within the camera’s view range when viewed from multiple viewpoints, and draw their locations.
My idea is to use raycaster.setFromCamera(twoDTarget, eachCamera) and gradually change the value of twoDTarget from (-1,-1) to (1,1) and assign the coordinates of each viewpoint to the value of eachCamera.
However, raycaster.intersect(model) returned strance locations like the below image.


Although this may be hard to see on the image, every intersect is in a rotated place.
Perhaps this is the problem derived from the gap between local- and world-coordinate, but I couldn’t find a solution after a trial and error.

Where should be fixed?
Here is the code:

<html>
  <head>
    <meta charset="utf-8" />
    <script src="https://unpkg.com/three@0.140.2/build/three.min.js"></script>
    <script src="https://unpkg.com/three@0.137.4/examples/js/controls/OrbitControls.js"></script>
    <script src="https://unpkg.com/three@0.137.4/examples/js/loaders/GLTFLoader.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/sprintf/1.1.2/sprintf.min.js"></script>
    <script src="js/utils/BufferGeometryUtils.js"></script>
  </head>
    <body>
    <canvas id="myCanvas"></canvas>
    <div>
    <input type="button" value="Check" id="check">
    </div>
    <script>
            
      function buttonClick(){
        init();
      } 

      function init() {
        const width = 960;
        const height = 540;

        const canvasElement = document.querySelector('#myCanvas');
        const renderer = new THREE.WebGLRenderer({
          canvas: canvasElement
        });
        renderer.setPixelRatio(window.devicePixelRatio);
        renderer.setSize(width, height);
        renderer.toneMapping = THREE.CineonToneMapping;

        const scene = new THREE.Scene();
        const camera = new THREE.PerspectiveCamera(45, width / height, 0.1, 10000);
        camera.position.set(1, 1, 1);

        var axes = new THREE.AxisHelper(25);
        scene.add(axes);

        const controls = new THREE.OrbitControls(camera, canvasElement);
        controls.target.set(0, 0, 0);
        controls.update();

        const directionalLight = new THREE.DirectionalLight(0xffffff);
        directionalLight.position.set(-1, -1, -1);
        scene.add(directionalLight);
        const ambientLight = new THREE.AmbientLight(0xffffff, 10);
        scene.add(ambientLight);


        const ISMaterial = new THREE.MeshLambertMaterial({ color: 0xff0000});
        ISMaterial.opacity = 0.15;
        ISMaterial.transparent = true;
        let originAndTargetArray = [[3,3,3,0,0,0],[-3,3,3,0,0,0],[3,-3,3,0,0,0],[-3,-3,3,0,0,0],[3,3,-3,0,0,0],[-3,3,-3,0,0,0],[3,-3,-3,0,0,0],[-3,-3,-3,0,0,0]]
                
        const headMaterial = new THREE.MeshLambertMaterial({ color: 0xffff03});
        headMaterial.opacity = 0.15;
        headMaterial.transparent = true;
        
        for (let i=0; i<originAndTargetArray.length; i++){
          const head = new THREE.Mesh(
          new THREE.SphereGeometry(0.5, 32, 32),
            headMaterial
          );
          head.position.set(originAndTargetArray[i][0], originAndTargetArray[i][1], originAndTargetArray[i][2]);
          scene.add(head);
        }
        
        let raycasterList = [];

        for (let iO=0;iO<originAndTargetArray.length;iO++){
          for (let i=0;i<height;i++){
            if (i % 50 != 0) {
              continue;
            }
            for (let j=0;j<height;j++){
              if (j % 50 != 0) {
                continue;
              } 

              const eachCamera = new THREE.PerspectiveCamera(45, width / height, 0.1, 10000);
              eachCamera.position.set(originAndTargetArray[iO][0], originAndTargetArray[iO][1], originAndTargetArray[iO][2]);
              const eachControls = new THREE.OrbitControls(eachCamera, canvasElement);
              eachControls.target.set(0, 0, 0);
              eachControls.update();
              const raycaster = new THREE.Raycaster();
              const twoDTarget = new THREE.Vector2();
              twoDTarget.x = (i / width) * 2 - 1;
              twoDTarget.y = (j / height) * 2 - 1;
              raycaster.setFromCamera(twoDTarget, eachCamera);
              console.log("rc",raycaster);
              raycasterList.push(raycaster);
            }
          }
        }
          
        const loader = new THREE.GLTFLoader();
        loader.load('./models/gltf/head/scene.gltf', (gltf) => {  
            model = gltf.scene;
            scene.add(model);
            const intersectSpheres = [];
            for (let i=0; i<raycasterList.length;i++){
                console.log()
                const intersects = raycasterList[i].intersectObject(model);
                if (intersects.length < 1){
                  continue;
                }
                intersectXyz = intersects[0].point;
                console.log(intersectXyz)
                const geometrySphereIS = new THREE.SphereGeometry(0.3, 10, 10);
                const geometryTranslatedIS = geometrySphereIS.translate(
                    intersectXyz.x, intersectXyz.y, intersectXyz.z
                );
                intersectSpheres.push(geometryTranslatedIS);
            }
            const geometryIS = THREE.BufferGeometryUtils.mergeBufferGeometries(intersectSpheres);
            const meshIS = new THREE.Mesh(geometryIS, ISMaterial);
            scene.add(meshIS);
            renderer.setAnimationLoop(() => {
                renderer.render(scene, camera);
            });

        });
      }
      const CheckBtn = document.getElementById('CheckBtn');
      let checkButton = document.getElementById('check');
      checkButton.addEventListener('click', buttonClick);

    </script>
    </body>
</html>

This GLTF can be downloaded here:

Kinda hard to help if hard to see what the problem is :smiling_face_with_tear: Could you share a video / live example of the issue?

Thank you for the suggestion.

This is the video example. Now can you see what the problem is?

(I wanted to share the live example but I don’t know a good service where I can upload my gltf file)

Ah, I understand now. Try removing buffer geometry merging - it doesn’t take into account transforms (so either remove the merging or apply transforms to geometries directly instead of to meshes.)

A couple points:

As far as I know… the “point” in the raycast hit result is in worldspace,

You can re-use the camera, controls and raycaster… no need to new one up each time through your loop:

just camera.position.set ( x, y, z);
controls.target.set( targetx, y, z );
controls.update()

… but I see where you’re going with your logic…
Overall… it looks correct to me… so I’m not sure why some of your spheres are placed wrong? (but the ones on the front of the model look correct?)

That all said, If you want to get random points on the surface of a mesh, you could also use the MeshSurfaceSampler…

https://threejs.org/docs/#examples/en/math/MeshSurfaceSampler

( Used in this example: three.js examples )

I put your code into a glitch to experiment with…
Im drawing the hits with an instancedMesh instead of generating a bunch of separate geometries,

edit: NVM i fixed it… we were missing an glb.scene.updateMatrixWorld(true); It was a stale matrix problem. I added that and the hits look correct.

Your code is at the bottom of script.js

1 Like

Close. :slight_smile: it was a stale matrix problem but not on the merging of the geometries, but on the loaded GLB itself!

Thank you for your advice.
I tried removing bffer geometry merging, but it didn’t work.
As manthrax said, it seems a stale matrix problem.
But very much appreciated, thank you!

Thank you for your advice.
Although I couldn’t open your code on the link and I had to do a little trial and error, finally it worked.

I added the line model.updateMatrixWorld(true); just below the scene.add(model);

The suggestion of MeshSurfaceSampler was also interesting for me.

This thread was very helpful. Thank you very much.

1 Like

Ya… the code takes a little while to run and the head model takes a bit to load. If you leave the window open for a few minutes it should load.
Keep in mind in glitch, you are downloading the model and stuff over the wire so it’s going to take a while to run the first time before it caches.
Glad the updateMatrixWorld(true) helped! cheers.

1 Like