[Help Please ! ] OrbitController panning and zooming on large scene

I am trying to visualize large-scale point cloud data and since it extends to a larger distance, I have put the camera at 200.

 camera = new THREE.PerspectiveCamera(
    75,
    window.innerWidth / window.innerHeight,
    0.1,
    20000
  );
  camera.position.z = 100;
  controls = new OrbitControls(camera, canvas);
 
  controls.minPolarAngle = 0;
  controls.maxPolarAngle = Math.PI;
  controls.rotateSpeed = 2;
  controls.zoomSpeed = 2;
  controls.panSpeed = 5;
  controls.update();
  proj = mat4.perspective(
    mat4.create(),
    (90 * Math.PI) / 180.0,
    canvas.width / canvas.height,
    0.1,
    20000
  );

The problem with this is that when i go near my scene or point cloud visualization output, zoom or drag or any other control movement is very slow.

This is not a performance issue ( just to make sure I am able to articulate the problem). My FPS is 60 FPS.

you can see I have set zoom speed 2 and panSpeed 5 as well but still it’s not helping.

I can not set a pan speed of 100 because when I zoom out, even at that time I want my control to work nicely with good drag to the top view of the point cloud.

this is my output.

Any help please !