Ah, now I have another hypothesis. You may be struggling with repeated unnecessary raycasting happening in event handling, since you have many transform controls enabled in the scene. Then rendering itself is not a bottleneck. I think it really shouldn’t be.