vector.project(camera) produces very different Y-axis position after onCameraUpdate

Questions
1

Hi All, I am trying to overlay DOM objects (<li>s) which track the position of objects in my 3D scene (reasoning explained below) and have found that the .project(camera) method of Vector3 gives wildly different results when it is called during inital loading and after onCameraUpdate (without significantly moving the camera).

I have one function to create and position the <li>s which is run as the scene is loaded, and another function to update the position of the <li>s onCameraUpdate. Both of these functions call the same positionMarker() function and pass in the same Object3D in order to calculate the position to which to set the <li>s. This works perfectly for positioning the <li>s on the X-axis, but the Y-axis position it gives is wrong. When the positionMarker() function is called during the initial loading, the Y-axis value is almost but not exactly double the value it should be. When it is called again onCameraUpdate, it is much closer to desired value, but still wrong and — more importantly — different to the result given during initial loading. It is also not half the value given during initial loading. This second value — while not precisely where I want it — is close enough to where it needs to be for me to be able to work with it, but I cannot live with the wildly wrong Y-axis positions upon first load.

I have made a simple jsfiddle version of my project to explain what I am talking about: DOM object positioned over Three.JS scene - JSFiddle - Code Playground. There you can see the red boxes positioned away from the red spheres when you first run the code. When you move the camera, the red boxes snap to a different position over the red spheres. Note that the red squares should be positioned with their bottom left corner in the middle of the sphere.

I know from logging out “vector.y” before and after it is projected using .project() that the problem is during that step. In my jsfiddle example, on initial load, the vector.y before projection = 1.5 and after projection = -0.6516126864206032. After onCameraUpdate, the vector.y before projection is still equal to 1.5 but the value after projection changes to -0.00010623945071439685. This accounts for the change in placement, but has left me scratching my head as to why this is happening and what to do about it.

A potentially related issue is that when you move the camera quickly, both the X and Y axis positions scatter away from where they should be. While it is very obvious in my jsfiddle example, it is less noticeable — but still present — when I view it outside of jsfiddle. I mention this because it may help diagnose the main issue and because fixing it would be nice, but I can live with this issue if need be.

My reasoning for wanting to position <li>s over my 3D scene is twofold. The first and main reason is for accessibility; I want the labels on my page to be real HTML text that can be resized by the user and fall-back to an ordered list for those who can’t see the 3D scene. Secondarily, using raycasting to identify when I am hovering over a marker seems unnecessarily expensive and complicated when :hover in my css does everything I need it to.

Any help tracking down what is going on, whether it is a mistake in my code, a bug in that method, or a technical limitation would be very much appreciated. Similarly, if any of you have an alternative means of achieving the same objective which doesn’t rely on the .project(camera) method then I’d love to hear it.

Many thanks in advance.

2

I managed to find some manner of a solution to this problem, which I will document here incase anyone in the future runs into a similar problem.

Upon further drilling down (and writing my own function to replace the vector.project() method), I discovered that the problem was caused by a discrepancy between the position of the origin specified with camera.position.set() and the position of the origin in the centre of the screen which is forced by OrbitControls (and other controls). Upon initial load, vector.project(camera) took in the camera as positioned and rotated by camera.position.set() and camera.rotation.set(), whereas what was rendered to screen took in the camera as positioned and rotated by OrbitControls, hence the discrepancy between the position of the <li>s and the position of the markers. When the scene is re-rendered onCameraUpdate, vector.project(camera) uses the same camera position and rotation as is rendered. If I make sure to set my initial camera position and rotation so that the origin is in the centre of the screen like is forced by OrbitControls then the marker position does not jump when the camera is first moved.

There is still some inaccuracy to the position given by vector.project() when the camera Is moved quickly. I don’t understand why this would be the case and may make a separate post about this in future if it becomes an issue for me.