Detecting click in objects accurately?

Hello! First of all, sorry Im a noob!
Im trying to detect a click on an object (brick as shown in the image), but the detection in not accurate at all. The model was imported from a glb file exported from blender. If I try to detect the click on a brick, it would get 3 bricks down the actual clicked brick.
Im doing a simple click detection with a raycaster like:

if (model) {
    const intersects = raycaster.intersectObjects(model.children, true);
    if (intersects.length > 0) {
      console.log('Clicked on:', intersects[0].object);
      intersects[0].object.material.color.set(0xff0000);
    }
  }
}

Why may this be happening? Does it have to do with the model? Or my code in three js.

Thank you!

Probably the way the raycaster is set up from the mouse cursor is where the problem is… can you post that code, or create a reproduction for us to look at in jsfiddle?