use camera-controls, all this will go away GitHub - yomotsu/camera-controls: A camera control for three.js, similar to THREE.OrbitControls yet supports smooth transitions and more features.
the problem is fundamentally that you have two pieces of code pulling on the camera, which is a race condition. orbit controls is a sealed control with very limited functionality, if it causes the root issue i would suggest you don’t hack around it. let cc solve it. it operates like orbit controls (but smoother due to unity damping instead of ugly lerp), and it can also be controled programatically.
here’s a react example of this but it’s the same in vanilla, all you need to call is
controls.setLookAt(position, target, true)