Just for my understanding: do you want to have the camera look away from the target?
If that’s the case, you might(!!) possibly achieve the same effect (that is: HACK OrbitControls without any code change) by providing a “fake” target which would have to be re-computed on every new frame though.
Please report back.