Help Needed with Raycasting in React Three Fiber and Three.js

Questions
, ,
1

Hi everyone,

I’m working with React Three Fiber and Three.js and am having trouble getting raycasting to work correctly on a 3D model. Specifically, I want to detect clicks on various parts of the model and trigger events, such as showing an alert or updating the UI.

I’ve set up a basic component with raycasting, but it’s not detecting clicks reliably on certain parts of the model. For example, when clicking on the engine heads of my model, the raycasting fails to register the clicks.

Here’s a simplified version of my component code:

import { OrbitControls, useGLTF } from "@react-three/drei";
import { Canvas, useThree, extend } from "@react-three/fiber";
import { useEffect, useState } from "react";
import * as THREE from "three";

const InteractionHandler = ({ scene }) => {
  const { gl, camera } = useThree();

  useEffect(() => {
    const raycaster = new THREE.Raycaster();
    const pointer = new THREE.Vector2();
    let nearestObject = null;

    const onClick = (event) => {
      pointer.x = (event.clientX / window.innerWidth) * 2 - 1;
      pointer.y = -((event.clientY / window.innerHeight) * 2 - 1);

      raycaster.setFromCamera(pointer, camera);
      const intersects = raycaster.intersectObjects(scene.children, true);

      if (intersects.length > 0) {
        console.log(intersects);
        nearestObject = intersects[0];
      }
    };

    window.addEventListener("click", onClick);

    return () => {
      window.removeEventListener("click", onClick);
    };
  }, [scene, camera]);

  return null;
};

const Model = () => {
  const { scene } = useGLTF("./engine.glb");

  return (
    <>
      <Canvas style={{ width: "100vw", height: "100vh" }} dpr={[1, 2]}>
        <ambientLight intensity={1} />
        <directionalLight intensity={0.5} />
        <OrbitControls panSpeed={3} rotateSpeed={3} />
        <primitive object={scene} />
        <InteractionHandler scene={scene} />
      </Canvas>
    </>
  );
};

export default Model;

And here is a description of the GLB model I’m using:

I’d appreciate any advice on why the raycasting might not be working as expected and how I can fix it. If there are any common issues or configuration settings I should check, please let me know!

Thanks in advance for your help!

2

In onClick console log the value of scene.children and see if the meshes are there.

3

pointer events are inbuilt, you don’t (and shouldn’t) create your own handlers as you gain nothing from them. all the calculations are also done for you and fiber makes it the fastest it can possibly be as it only raycasts stuff that has actual handlers on them.

<primitive object={scene} onClick={e => {
  e.stopPropagation()
  console.log(e)
}} />

the e object will contain the dom pointer event data, the three event data and r3f extras. everything you need is in there. it supports all common pointer events, bubbling, stop propagation (if you want the closest object to the camera for instance) and pointer capture (if you want to drag stuff and the pointer may leave the window), see Events - React Three Fiber

if you want specific actions on specific meshes, this is what GitHub - pmndrs/gltfjsx: 🎮 Turns GLTFs into JSX components is for.

1 Like
4

PS your code has issues with separation of concern. the model component should hold a model, not a canvas. you also don’t need to give canvas 100vw/100vh, it stretches ootb to 100% of its parent. basic reset css styles take care of it.

* {
  box-sizing: border-box;
}

html,
body,
#root {
  width: 100%;
  height: 100%;
  margin: 0;
  padding: 0;
}

function App() {
  return (
    <Canvas>
      <ambientLight intensity={Math.PI} />
      <directionalLight intensity={0.5} />
      <OrbitControls panSpeed={3} rotateSpeed={3} />
      <Engine />
    </Canvas>
  )
}

function Engine() {
  const { scene } = useGLTF("./engine.glb")
  const click = (e) => {
    e.stopPropagation()
    console.log(e)
  }
  return <primitive object={scene} onClick={click} />
}

here’s a super basic sandbox that explores some of these concepts https://codesandbox.io/p/sandbox/gifted-varahamihira-rrppl0y8l4?file=%2Fsrc%2FApp.js%3A18%2C32

1 Like
5

Thank you for the suggestion and pointing out the issues with separation of concern !

I tried the approach using the built-in pointer events, and while it works in some cases, I’m still facing a couple of issues:

  1. Inconsistent Event Detection: The raycasting seems to work only about 1 out of 5 times. For example, if I click the same object 5 times, the event only triggers once. The other 4 clicks don’t seem to register at all, and I’m not sure why the detection is so inconsistent. This happens especially when rotating the model or interacting with certain parts.
  2. Certain Objects Not Registering: There are parts of the model where clicking doesn’t trigger any console logs or event data at all. It seems like raycasting isn’t being applied to those objects, though they should be part of the same scene.

I’ve made sure that the event handlers are set up correctly, but it feels like something is missing or not working as expected, especially for certain parts of the model. Any ideas on what could be causing this behavior?

Thanks again for your help!

6

I have hosted the project demo in a codesandbox here for better collaboration:
Demo

Thanks in advance!.

7

yes I checked, the meshes are available.