Honestly, this sounds like “I want to sail to the other side of the lake with a boat, but I want the boat to be stationary and the lake should move instead of the boat”.
Just kidding. If you want to keep the scene stationary, and manipulate only the camera, this is some specific combination of translation and rotation - the camera should orbit the scene, but should look away from the point of rotation.
One approach is to use manual control of the camera. Here is a demo with some custom rudimentary navigation (panning and rotating, no zooming, no damping, no other extras). For code simplicity the camera is placed inside another object (called cameraHost) just to reuse its rotation. You can avoid a host and use a virgin camera, but you have to do the calculations in your code.
https://codepen.io/boytchev/full/KwdoVrR
Good luck with your project.