Raycasting not accurate in immersive fullscreen AR

Questions
,
1

I’m working on an AR app and I’ve noticed that raycasting is not accurate on a Pixel 4. It’s especially noticeable in landscape mode. I want to be able to tap on the screen to place objects.

const screenPosition = normalizedDeviceCoordinates(event.clientX, event.clientY)
this.raycaster.setFromCamera(screenPosition, this.camera)
const hits = this.raycaster.intersectObjects(this.scene.children)

I am updating the camera’s aspect and renderer’s size when the screen size changes though I’m not sure if it’s necessary since WebXRManager might be doing it.

Setting the camera to renderer.xr.getCamera().cameras[0] improves it but it’s still not perfect. I think what is happening is that the virtual camera doesn’t match the physical camera on the phone but I don’t know how to fix it.

2

The screenPosition code looks strange.
Or should I say: it doesn’t match anything I’ve seen in three.js example demonstrating touch/mouse raycasting.

It’s just a guess tho
To be honest I never used normalizedDeviceCoordinates and have no idea if this function output same result as the commonly used screen/space formula.

3

That’s just a function I created myself which uses the standard Three.js code. I should have inlined it.

const screenPosition = new Vector2((event.clientX / window.innerWidth) * 2 - 1, -(event.clientY / window.innerHeight) * 2 + 1)
this.raycaster.setFromCamera(screenPosition, this.camera)
const hits = this.raycaster.intersectObjects(this.scene.children)

Doing a few more tests I find that if I set the camera’s field of view to 60 in portrait and 30 in landscape it kind of works though it’s probably quite device specific. I was hoping that Three.js/WebXR would take care of these details but it doesn’t seem to be the case.

4

Hi, I just noticed the same issue

to fix it use the first camera of the xr cameras while in AR: via renderer.xr.getCamera().cameras[0]