Here is one ugly, truly ugly solution (so ugly, that looking at the source may cause brain damage). The solution works only for canceling rotation damping. Canceling panning and zooming requires a little bit more work.
https://codepen.io/boytchev/full/qBJxdxo
I think the best would be to modify OrbitControls.js by adding a function to clear sphericalDelta and a few other variables.