WebXR + Reflection => strange camera matrix

Hi all
I am working on WebXR teleportation.

Everything worked fine until I add water to the scene. (“three/examples/jsm/objects/Water.js”)

This is how teleportation works

let cameraRig = new THREE.Group();
cameraRig.add(camera);
scene.add(cameraRig);


function teleport() {
    cameraRig.position.add(new THREE.Vector3(10, 10, 10));

    console.log(`camera.matrix : ${cameraRig.children[0].matrix.elements}`);
    console.log(`camera.matrixWorld : ${cameraRig.children[0].matrixWorld.elements}`);
}

And this is what I get in console

camera.matrix : 1,0,0,0,0,1,0,0,0,0,1,0,10,11,10,1 
camera.matrixWorld : 1,0,0,0,0,1,0,0,0,0,1,0,20,21,20,1 

What I should get is

camera.matrix : 1,0,0,0,0,1,0,0,0,0,1,0,0,1,0,1 
camera.matrixWorld : 1,0,0,0,0,1,0,0,0,0,1,0,10,11,10,1 

Order to reproduce error

  1. Enter XR
  2. Press t to teleport and check console
  3. Press w to toggle water (remove/add)
  4. Press t to teleport and check console

I have tested it with
Win10 + Chrome + Webxr simulator plugin WebXR API Emulator - Chrome Web Store
Oculust Quest 2 + Oculus Browser

I have found a walk around but it feels unsafe. ( removing scene.add(cameraRig) )
I suspect Water’s onBeforeRender method.

Why does this thing happen? Will it be fixed?

1 Like

I assume some people might not comfortable with react, so I updated Codesandbox demo to plain javascript. And removed all dependencies other than three.js.

I guess the bug is caused because some of w3c webxr standards are not compatible with three.js ??

Will this problem vanish when webGPU and WGSL replace old regacies??

Any thoughts would be thankful!

Did you test with native WebXR to make sure it is not a problem with the emulator extension? (I haven’t verified your bug yet)

1 Like

@trusktr
I have tested it with Oculus Quest2 + Oculus Browser

chrome://inspect/#devices

codesandbox example running and its console

I am quite sure this is not a bug from WebXR Emulator since I found this bug using Oculus Quest2 with Oculus Link

I rounded the camera.matrix and the camera.matrixWorld in console for better readability.

A youtube video that shows how to produce the same error.

Seems like a “bug” in the design of Water.js. It is not accounting for how an end user may use a camera.

Here you can see it does a bunch of stuff with the camera that you pass into WebGLRenderer:

Something is going on there.

1 Like

@trusktr
Thank you for the feedback.

I have played around with Water.js and I have found something.

  1. If I comment out this line everything works fine
  1. Water’s onBeforeRender method uses the camera to make water reflection. It seems it does not mutate the camera

  2. This bug only occurs in XR, I suspect WebXRManager’s updateCamera function.

I will let you know If I find more.

I have found that it is not only happening with Water.js, it also happens with Reflector.js (three/examples/jsm/objects/Reflector.js)

I changed codesandbox demo’s water to reflector

I think I found the source of the bug.

In WebXRManger, updateCamera function gets called every render and it checks whether the camera has a parent.

Reflector’s onBeforeRender method renders reflection with a virtual camera. Maybe that virtual camera has something to do with updateCamera function.

It is not clear to me why this behavior is needed and how to solve this problem. :thinking:

I think you’re into something. It seems like an oversight in how those example classes were designed and/or how WebXRManager is designed. I think this is worthy of a bug report on the GitHub repo.

1 Like

@trusktr
Thank you for the feedback.

After few more research I will create an issue on the Github repo.

I used to be Unreal Engine 4 Developer.
I think this bug is occurred because I tried to implement functions in UE4 way.

It is common sense that “Player” object has a camera as its children. (So called ECS, Entity component system)

But WebXR may needs different approach and/or solution. It is not built to be game engine like UE4.

Again , Thank you @trusktr

2 Likes

In fact making the camera a child of any object is completely valid in Three.js, so this may be considered a bug because in this case making the camera a child of an object, while using a class like Water or Reflector, introduces the issue. Those are classes from the examples folder though, so it might not be high priority.

1 Like