PointCloud intersection raycasting problem

Good morning,
I am trying to paint over a pointcloud, so I used the PCD loader provided in examples and I am returning a Point object.
I am following the tutorial to get the intersection with the raycast on the point cloud and display the red sphere there which is fine. However the sphere is always a bit before the points and if I try to use the intersections of the rays with the pointcloud and set the closest ones to the color black using their indices, always half of my point cloud on the same size becomes black but never the points close to where the mouse is.
I do the intersection as follow:

raycaster.setFromCamera( mouse, camera );

  		if(pointcloud != null){
  			intersects = raycaster.intersectObject(pointcloud);
  			intersection = ( intersects.length ) > 0 ? intersects[ 0 ] : null;
  			if(PAINTING){
  				console.log("painting");
  				var geometry = pointcloud.geometry;
  				var attributes = geometry.attributes;
  				// console.log(attributes);
  				if ( intersects.length > 0 ) {
  					attributes.color.array[intersection.index] = 0.0;
  					attributes.color.array[intersection.index+1] = 0.0;
  					attributes.color.array[intersection.index+2] = 0.0;
  					attributes.color.needsUpdate = true;
  				}
  			}
  		}

And here what I get after trying to paint points on the right of the strawberry and bottom:


Any idea ?

Hi!
Any chance to provide a working live code example?

As I am importing quite a few three js files and the point cloud, I uploaded an archive with the code.
The html file is in examples/Annotation.html