This is a very minor/unimportant question but shouldn’t Vector3.unproject be Vector2.unproject?
Because you pass only 2 variables (x and y of pointer on screen) to it like new THREE.Vector3(x, y, 0).unproject(camera), so why have z with always 0?
Well, it doesn’t have to be 0.
The transform happens from normalized frustum coordinate space back into the world space. What that means is if you set z to 0, you’ll get a point in the world that is at the midpoint between two clipping planes in the normalized space.
If you set it to -1 if will get you near clipping plane and if you set it to 1 if you will get the point at far clipping plane. Usually these z coordinates are largely meaningless as we use them to compute a ray direction.
Also, the method is a bit complex in its proper description. What it does is that it applies reverse sequence inverse matrix projections that are applied to geometry vertices for rendering. That is - it undoes camera’s projection matrix, followed by undoing camera’s transform.
For intuition, here’s what the simplest vertex shader will look like:
gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );
Where position is geometry’s vertex position, projectionMatrix is the camera’s projection matrix and modelViewMatrix.
So the Vector3.unproject is basically doing the opposite transform. Getting you from gl_Position to world_position where world_position = modelMatrix * position.
I’m simplifying a lot of things here, like the fact that 4x4 matrix projections are done on 4d vectors instead of 3d ones. I’m sure someone else could chime in with more detail also.
In short - there are some cases where it might be useful to use z values other than 0.
Thanks! Great answer!