How do you make html position follow a mesh in react three fiber/drei?

I would like for a button, when it moves down, the html element acting as a label for each button to follow it. So far the buttons are moving without the labels.

Here is my code so far:

export function CalculatorButton({id, char, position, rotation, scale, currentButtonPressed, setCurrentButtonPressed}) {
  const group = useRef()
  const instances = useContext(context)
  const { materials, animations } = useGLTF('./3D_Assets/calculator_button-transformed.glb')
  const { actions } = useAnimations(animations, group)

  const symbolMaterial = new MeshStandardMaterial({
    color: new THREE.Color( 0x003986 ),
    side: THREE.DoubleSide,
  })

  const symbolPosition = new THREE.Vector3(0, .697, 0)

  useEffect(() => {
    actions['Button_Press'].play(repetitions)
    return () => {}
  }, [currentButtonPressed]);

  return (
    <group 
      ref={group} 
      dispose={null} 
      position={position} 
      rotation={rotation} 
      scale={scale} 
      onPointerDown={(char) => setCurrentButtonPressed(char)}
    >
      <Html
        transform
        position={symbolPosition}
        rotation={char === "←" ? [-Math.PI/2, 0, -Math.PI/2] : [-Math.PI/2, 0, 0]}
        scale={char === "←" ? [1, .75, 1] : [1, 1, 1]}
        material={symbolMaterial}
        occlude
        // color="#003986"
      >
        <p>{char}</p>
      </Html>
      <group name="Scene">
        <group name="Button">
          <instances.ButtonRim />
          <instances.ButtonBody />
        </group>
      </group>
    </group>
  )
}

useGLTF.preload('./3D_Assets/calculator_button-transformed.glb')

If you have any suggestions, please let me know.
Thank you!

By simply nesting it. If that were a plain mesh and not html it would not move either if a sibling group moves.

Ps don’t destruct position, scale etc, just do …props and then spread it {…props}, the component will be more flexible