Select from @react-three/drei not detecting meshes on click

I have a Canvas with a grid and few meshes (square, polygon, some lines) that I render on top of it. I try to use Select from @react-three/drei to detect clicked meshes. When I’m in a 2D top down mode (when is3DModeEnabled is false), Select from @react-three/drei is able to detect the clicked meshes which are on top of the gridHelper. However, when i toggle the camera to 3D mode, the Select only logs the gridHelper. I tried to call updateProjectionMatrix, but it doesn’t help and I still can’t detect my meshes with Select when clicking on them in 3D mode.

What could the problem be?

Any ideas / advice would be appreciated as I’ve been banging my head for the last couple of days on this problem.

The setup looks like this:

import { Select } from "@react-three/drei";

...

      <Canvas dpr={dpr}>
        <Select
          onChange={(e) => {
            console.log(e);
          }}
        >
          <Camera />

          <Grid canvasRef={canvasContainerRef} scale={10}></Grid>
        </Select>
      </Canvas>

Camera:

  useEffect(() => {
    if (!is3DModeEnabled) {
      setOrtographicCameraPosition([0, 90, 0]);
    } else {
      setOrtographicCameraPosition([50, 50, 50]);
    }

    defaultCam.current.updateProjectionMatrix();
  }, [is3DModeEnabled]);

    <>
      <directionalLight position={[10, 10, 5]} intensity={5} />
      <directionalLight position={[-10, -10, -5]} intensity={1} />
      <OrbitControls makeDefault enableRotate={false} />
      <OrthographicCamera
        ref={defaultCam}
        updateDefaultCamera={false}
        makeDefault
        position={ortographicCameraPosition}
        {...cameraProps}
      />
    </>

And the Grid component looks like this:

<>
   <line geometry={polylineGeometry}>
      <lineBasicMaterial attach="material" color="red" />
      </line>

      {renderedMeshes}

      {renderedCompletedLines}

      {renderedPolylines}
      <group scale={scale}>
          <gridHelper ref={gridRef} args={[1, divisions, "#2A303C", "#2A303C"]} />
          <mesh
            receiveShadow
            ref={planeRef}
            rotation-x={-Math.PI / 2}
            onPointerMove={onPointerMove}
            onPointerDown={onPointerDown}
            onDoubleClick={onDoubleClick}
          >
            <planeGeometry args={[scale, scale, divisions, divisions]} />
            <meshStandardMaterial
              transparent
              color="#0C1425"
              polygonOffset
              polygonOffsetUnits={1}
              polygonOffsetFactor={1}
            />
          </mesh>
      </group>
</>