I beg to disagree:
the implementation of contradicting requirements is not “simple”. You need to provide some cue to your app as to which interpretation of user input you want at any given moment, wether it’s camera control or object control.
Of course it helps, to first undergo the mental gymnastics required to get an understanding of off-center rotations in 3D space. What you call “fiddling” is just a consequence of someone trying to take shortcuts, without the proper understanding.