How can I add annotations to different mesh parts or textures and shift the camera focus to them when clicked?

vakaer · May 6, 2025, 12:59pm

Hi everyone,

I’m working with a 3D model in a React Three Fiber project and want to add annotations that are attached directly to different parts of the model — ideally specific meshes or even textures. So far, I’m rendering annotations using the <Html> component from @react-three/drei, positioned manually using [x, y, z], but they float above the model instead of sticking to its surface.

What I want to achieve is:

Attach annotations to specific mesh parts or surface positions on the model so they move naturally with camera orbit.
Focus or shift the camera (using OrbitControls or another method) to that annotation when it’s clicked.

Has anyone implemented something similar?
What would be the best way to bind annotation positions to mesh geometry and make the camera transition smoothly to the annotation point when selected?

Any code examples, libraries, or logic to help with mesh-based annotations and camera transitions would be much appreciated!

Thanks in advance.

I want to make something similar to this model
https://sketchfab.com/3d-models/heart-superficial-anatomy-26aca9f964d446c5a83ec6a8cda5ea3b

seanwasere · May 6, 2025, 1:12pm

Something like R3F House with Annotations

vakaer · May 7, 2025, 8:18am

Hi, I’ve gone through this example, but doesn’t matter which position I apply to the annotation, it doesn’t render above the mesh, rather the annotations are being rendered above the model instead of being on top of the mesh. I can’t figure out what i’m missing.

import { Html } from '@react-three/drei'
import { useState } from 'react'

export interface Annotation {
  id: number
  title: string
  description: string
  targetNodeName: string;
  position: [number, number, number]
  camPos: {
    x: number;
    y: number;
    z: number;
  }
  lookAt: {
    x: number;
    y: number;
    z: number;
}
}

export const annotations: Annotation[] = [
  {
    id: 1,
    title: 'Left auricle',
    targetNodeName: 'left-auricle',
    description: 'Ear-like flap overlying the left atrium',
    position: [-0.5, 1.2, 0.3],
    camPos: { x: 1, y: 100, z: 1 },
    lookAt: { x: -0.5, y: 1.2, z: 0.3 }
  },
  {
    id: 2,
    title: 'Right auricle',
    targetNodeName: 'right-auricle',
    description: 'Ear-like flap overlying the right atrium',
    position: [0.6, 1.1, 0.4],
    camPos: { x: 1, y: 1, z: 1 },
    lookAt: { x: 0.6, y: 1.1, z: 0.4 }
  }
];

interface AnnotationsProps {
  onSelectAnnotation: (annotation: Annotation) => void;
}

export const Annotations = ({ onSelectAnnotation }: AnnotationsProps) => {
  const [selected, setSelected] = useState<Annotation | null>(null)

  const handleClick = (annotation: Annotation) => {
    setSelected(annotation)
    onSelectAnnotation(annotation)
  }

  return (
    <>
      {annotations.map((annotation, index) => {
        return (
          <Html key={index} position={[annotation.lookAt.x, annotation.lookAt.y, annotation.lookAt.z]}>
            <svg onClick={() => handleClick(annotation)} height="34" width="34" transform="translate(-16 -16)" style={{ cursor: 'pointer' }}>
              <circle
                cx="17"
                cy="17"
                r="16"
                stroke="white"
                strokeWidth="2"
                fill="rgba(0,0,0,.66)"
              />
              <text x="12" y="22" fill="white" fontSize={17} fontFamily="monospace" style={{ pointerEvents: 'none' }}>
                {index + 1}
              </text>
            </svg>
            {annotation.description && annotation.id === selected?.id && (
              <div className='relative'>
                <div id={'desc_' + index} className="annotationDescription" dangerouslySetInnerHTML={{ __html: annotation.description }} />
                <button onClick={() => setSelected(null)} className="absolute top-1 right-2 text-white text-md">close</button>
              </div>
            )}
        </Html>
        )
      })}
    </>
  )
}

import { Text, useGLTF } from "@react-three/drei";
import { useLayoutEffect, useRef } from "react";
import { JSX } from "react/jsx-runtime";
import * as THREE from 'three';
import { GLTFParser } from "three/examples/jsm/Addons.js";

export interface GLTF {
  animations: THREE.AnimationClip[]
  scene: THREE.Group
  scenes: THREE.Group[]
  cameras: THREE.Camera[]
  asset: {
    copyright?: string
    generator?: string
    version?: string
    minVersion?: string
    extensions?: any
    extras?: any
  }
  parser: GLTFParser
  userData: any
}

export type ObjectMap = {
  nodes: {
    [name: string]: THREE.Object3D
  }
  materials: {
    [name: string]: THREE.Material
  }
}

export function Heart(props: JSX.IntrinsicElements['group']) {
  const groupRef = useRef<THREE.Group>(null);
  const { nodes, materials }: ObjectMap = useGLTF('/heart_superficial_anatomy.glb')

  useLayoutEffect(() => {
    if (groupRef.current) {
      const box = new THREE.Box3().setFromObject(groupRef.current)
      const center = box.getCenter(new THREE.Vector3())
      groupRef.current.rotation.x = Math.PI
      groupRef.current.position.sub(center) // shift entire group so it's centered
    }
  }, [])

  const getMeshName = (object: THREE.Object3D): string | null => {
    const mesh = object as THREE.Mesh;
    if (mesh.material) {
      const material = mesh.material as THREE.MeshBasicMaterial;
      return material.name;
    }
    return null;
  };
  return (
    <group ref={groupRef} {...props} dispose={null} scale={0.01}>
      <group position={[0, 0, 5]} rotation={[0, 0, 0]}>
        {/* {Object.entries(nodes).map(([key, value]) => {
          const mesh = value as THREE.Mesh;
          const material = mesh.material as THREE.MeshBasicMaterial;
          console.log("🚀 ~ {Object.entries ~ material:", material)
          return (
            mesh.isMesh && (
              <mesh
                key={key}
                name={material.name}
                geometry={mesh.geometry}
                material={material}
                castShadow
                receiveShadow
                onPointerOver={(e) => {
                  e.stopPropagation();
                }}
                onPointerOut={(e) => {
                  if (e.intersections.length === 0) {
                    console.log('e.intersections.length', e.intersections.length)
                  }
                }}
                onClick={(e) => {
                  e.stopPropagation();
                  const name = getMeshName(e.object);
                  if (name) {
                    console.log('e.object.uuid', e.object.id)
                  }
                }}
              />
            )
          );
        })} */}
        <mesh
          castShadow
          receiveShadow
          geometry={nodes?.Object_2?.geometry}
          material={materials?.texture}
        />
        <mesh
          castShadow
          receiveShadow
          geometry={nodes?.Object_3?.geometry}
          material={materials?.texture}
        />
        <mesh
          castShadow
          receiveShadow
          geometry={nodes?.Object_4?.geometry}
          material={materials?.texture}
        />
        <mesh
          castShadow
          receiveShadow
          geometry={nodes?.Object_5?.geometry}
          material={materials?.texture}
        />
        <mesh
          castShadow
          receiveShadow
          geometry={nodes?.Object_6?.geometry}
          material={materials?.texture}
        />
        <mesh
          castShadow
          receiveShadow
          geometry={nodes?.Object_7?.geometry}
          material={materials?.texture}
        />
        <mesh
          castShadow
          receiveShadow
          geometry={nodes?.Object_8?.geometry}
          material={materials?.texture}
        />
        <mesh
          castShadow
          receiveShadow
          geometry={nodes?.Object_9?.geometry}
          material={materials?.texture}
        />
        <mesh
          castShadow
          receiveShadow
          geometry={nodes?.Object_10?.geometry}
          material={materials?.texture}
        />
        <mesh
          castShadow
          receiveShadow
          geometry={nodes?.Object_11?.geometry}
          material={materials?.texture}
        />
        <mesh
          castShadow
          receiveShadow
          geometry={nodes?.Object_12?.geometry}
          material={materials?.texture}
        />
      </group>
    </group>
  )
}

useGLTF.preload('/heart_superficial_anatomy.glb');

seanwasere · May 7, 2025, 10:12am

Without looking into to much detail. Maybe your annotations are being positoned relative to their parent group position. This is the difference between local and global positions.

<group ref={groupRef} {...props} dispose={null} scale={0.01}>
      <group position={[0, 0, 5]} rotation={[0, 0, 0]}>
        {/* {Object.entries(nodes).map(([key, value]) => {

try chainging position={[0, 0, 5]} to position={[0, 0, 0]},
or delete it,
or do you even need to add this group into your code,
or put the group outside the parent group.

Experiment a bit.

In my example, the annotations are added to the main scene, not a child in it that already may have some transforms applied.

vakaer · May 7, 2025, 11:38am

I did some experimentation in the annotations and the Model, I can position the annotation but it’s still not relative to the part of the model where i want to add annotation. Here’s the attached video and screenshot.

Code Sandbox: https://codesandbox.io/p/sandbox/heart-with-annotations-qltw9f

seanwasere · May 7, 2025, 5:13pm

You can animate the orbit controls target movement so that it doesnt snap.

Your codesandbox didnt work so i made my own version the way i would do it.

It’s built quickly based on my R3F house example and not completed. If you wanted more, i’d have to charge.

Topic		Replies	Views
Annotation Text attached to part of 3d model Questions geometry , mesh , text , scene , labels , annotations , scenechildren , save	3	1177	January 14, 2024
How to add marker on specific mesh? Questions labels , react-three-fiber	1	1581	April 23, 2022
How to make annotations move with the 3D model Questions labels , annotations , hotspots	14	10522	August 17, 2023
Dynamic annotations on Obj loaded model Questions obj-loader , annotations , react-three-fiber	4	125	July 26, 2024
Dynamic create annotation issue Questions	7	2072	February 12, 2019

How can I add annotations to different mesh parts or textures and shift the camera focus to them when clicked?

Related topics