Webxr camera is not at position of perspective camera

Pravin_Poudel · November 20, 2022, 4:21am

My webxr camera is not at position of camera given in three.js perspective camera used in render function !!

I found somewhere that the way to make webxr camera appear in same position in the camera is this so I added this in the animate function

const xrManager = renderer.xr
let firstTime = true
function animate(now: number) {
    let delta = clock.getDelta()
    delta = Math.max(delta, 0.1)
    world.step(delta)
    if (ballAdded) {
        balls.forEach((element, index) => {
            element.position.set(
                ballBodies[index].position.x,
                ballBodies[index].position.y,
                ballBodies[index].position.z
            )
        })
    }
    if (orbitControls.enabled) {
        orbitControls.update()
    }
    

    cannonDebugRenderer.update()
    TWEEN.update()
    KeyBoardHandler.keyUpdate(handlers, keys, delta * 1000)
    if (character) {
        character.position.copy(camera.position)
    }
    
    if (xrManager.isPresenting && firstTime) {
        firstTime = false
        const baseReferenceSpace = xrManager.getReferenceSpace(),
            offsetPosition = camera!.position,
            offsetRotation = camera!.rotation
        console.log(baseReferenceSpace)
        const transform = new XRRigidTransform(
                { x: offsetPosition.x, y: offsetPosition.y, z: offsetPosition.z, w: 1 },
                {
                    x: offsetRotation.x,
                    y: -1 * offsetRotation.y,
                    z: offsetRotation.z,
                    w: 1,
                }
            ),
            //const transform = new XRRigidTransform( offsetPosition, { x: offsetRotation.x, y: -(offsetRotation.y - 0.5) , z: offsetRotation.z, w: offsetRotation.w } ),
            teleportSpaceOffset = baseReferenceSpace!.getOffsetReferenceSpace(transform)

        xrManager.setReferenceSpace(teleportSpaceOffset)
    }
    updateControllerAction()
    render(delta)
}


function render(delta) {
    // renderer.setViewport(0, 0, window.innerWidth, window.innerHeight)
    // composerScreen.render(delta)
    // renderer.clear(false, true, false)
    // renderer.setViewport(20, window.innerHeight - 256, 256, 256)
    // composerMap.render(delta)
    renderer.render(scene, camera)
    labelRenderer.render(scene, camera)
    // scene.background = new THREE.Color(0xffffff)
}


renderer.setAnimationLoop(animate)

but the problem is that camera scene is this:

but after adding that code camera render this:

I guess this is either outside of my mesh

when i go out without VR mode, this is my mesh outside

after changing camera position to somewhere and positioning VR headset in webxr emulator for so many time,

I managed to get this manually

Is there anyway i can have webxr camera at the camera position !!!

seanwasere · November 20, 2022, 8:06am

You add your camera to a group, then add the group to the scene, and position the group where you want your head to be.
Thats what I do in these two projects

TeleportVR : GitHub - Sean-Bradley/TeleportVR: Teleport module for WebVR, THREE.js and VRController.js projects
Ball-VR : GitHub - Sean-Bradley/Ball-VR: A ball rolling experiment in VR

For TeleportVR, see lines 23-26 : TeleportVR/teleportvr.ts at b076678b3571e20c53099fc2a5b369b91de626e5 · Sean-Bradley/TeleportVR · GitHub

For Ball-VR, it’s a little more complicated.
It’s all in this script: Ball-VR/ball.ts at dbc4c2747842d3ed20942c2013393b276b563754 · Sean-Bradley/Ball-VR · GitHub
The camera is added to a pivot, pivot add to ball, chasecam added to pivot, chasecam positioned offset from pivot, and then it can rotate from an extended viewpoint similar to the official orbit controls.

Pravin_Poudel · November 20, 2022, 8:23am

ohh wow, it seems its totally different thing !!

I did this:

this is my addcamera function

function addCamera() {
    const camera = new THREE.PerspectiveCamera(
        90,
        window.innerWidth / window.innerHeight,
        0.1,
        1000
    )
    camera.position.set(50, 100, 50)
    camera.lookAt(0, 1, 15)
    return camera
}

and I after I get camera from calling this function I addeded it into group and position it

camera = addCamera()
 let group = new THREE.Group()
 group.add(camera)
 group.position.set(50, 100, 50)
 scene.add(group)

the now it is rendering as this:

Which is as expected !!

Thank you so much !!

I am feeling kind of lost in webxr because for three.js, I was totally relied in your website but for webxr based on three.js there seems not such resource.

I don’t understand why positioning camera did not work but positioning group with camera as only child worked !!

and one morething, I don’t see my controller now in space which was visible when I was not doing correct, is it expected to be visible ?

because I have written code to create ray when button is pressed but now it is not even when button is pressed

Update:

After changing mesh scale size from 100 to 1 inside the loader by

_child.scale.set(1, 1, 1)

and removing camera position set

camera.position.set(50, 100, 50)

from the code, i don’t know if we need it

and positioning the group

let group = new THREE.Group()
group.add(camera)
group.position.set(0.5, 0.75, 0.5)

to 0.5, 0.75, 0.5

i can see this:

Thank you so muchbut they are not properly aligned !!

Do i have to do something so that they would be properly aligned with camera

I am on webxr emulator,

and controller is far when I move headset back

close when I move controller forward

seanwasere · November 20, 2022, 10:24am

I have lots or VR resources on my website. Plus Ball-VR.

Official examples are also very good. https://threejs.org/examples/?q=webxr

You can reset the coordinates of your headset by long pressing using one of the controller buttons. I can’t remember which one.
Also, the scale of your model is probably not 1 unit = 1 metre. You will need to rescale it using whatever method.
I would rescale it in blender, and estimate based on some real-world measurement, such as the chair height. It would be an estimate and not exactly correct, but close enough for me.

Pravin_Poudel · November 20, 2022, 10:37am

Can i ask why u went with class with vr code ?

Is there any specific reason for this?

jrjdavidson · November 20, 2022, 10:00pm

I’ve been following this conversation with interest! This made me wonder, do you add the controllers to the scene, or to the new ‘camera group’ you just created. Do your controllers get place in the right position if you add them to the group?

Pravin_Poudel · November 20, 2022, 11:12pm

hi, thank you !!

When I add the controller and controller grip into the camera group, they position at the same height as the camera and if I add those two into the scene they stay at the original position as in posted pic.

When I drag the camera, they don’t move with it, and most importantly left one is almost hidden !!!

this is my controller code:

PS: I changed the variable group that store the camera to variable name “user” just to avoid the group identifier as variable name itself.

So user is the

camera = addCamera()
    user.add(camera)
    user.position.set(0.0, 1.0, 0.0)
    scene.add(user)


function buildControllers() {
    const controllerModelFactory = new XRControllerModelFactory()
    const controllers: Array<any> = []

    const rayGeometry = new BufferGeometry().setFromPoints([
        new THREE.Vector3(0, 0, 0),
        new THREE.Vector3(0, 0, -1),
    ])
    const line = new Line(
        rayGeometry,
        new THREE.LineBasicMaterial({
            color: 0x333300,
            linewidth: 5,
        })
    )
    line.scale.z = 0

    for (let i = 0; i < 2; i++) {
        const _controller = renderer.xr.getController(i)
        _controller.add(line.clone())
        _controller.name = 'controller' + (i + 1)
        _controller.userData.selectPressed = false
        _controller.userData.selectPressedPrev = false
        // add ray in it
        user.add(_controller)
        controllers.push(_controller)
        const grip = renderer.xr.getControllerGrip(i)
        grip.add(controllerModelFactory.createControllerModel(grip))
        user.add(grip)
    }
    return controllers
}

and output is like this now:

which is atleast better than adding in scene becase it seems it went at level of camera but i dont think that is also what is required (i dont know)

BVut when i move the headset in emulator it doesnot move with it

jrjdavidson · November 20, 2022, 11:26pm

Just FYI, I had similar issues the first time that I tried using webXR, inlcuding camera positioning, getting the controllers in the right place etc. I was able to solve all those problems by shifting the world 0,0,0 coordinates to where I wanted the user to start when beginning a webxr session ( by shifting the model). The only problem I still have is that the VR view starts at 90 deg to the initial camera view.

I’m keen to know if you are able to solve all these problems though, I think it’s probably worth the effort.

Pravin_Poudel · November 20, 2022, 11:30pm

how do you make the controller follow the camera?
Am I supposed to add a controller to the camera group or in scene?

Pravin_Poudel · November 20, 2022, 11:42pm

and do you have an idea or thinking of why does it work when we add a camera to the group but not when the camera is directly added to the scene?

jrjdavidson · November 20, 2022, 11:55pm

I had some issues when I played with offsetting by attaching the camera to a group, so now I just add the controllers to the scene, and the camera is unattached. However the suggestions from others in the forum might solve these issues. Have you tried testing the code with an actual headset?

I’m guessing that by ‘it work’ you mean that you get the same position as the camera in and out of VR? I believe the reason is that the VR camera(s) are positioned at 0,0,0 when you start the sessions and in your group example the camera outside VR is also at 0,0,0 in the group space.

Pravin_Poudel · November 20, 2022, 11:57pm

No I dont have VR headset

Pravin_Poudel · November 21, 2022, 12:00am

No In my case,

I am outside the house at bottom of house in non-vr mode.

But when i click enter VR, i get inside house in this position

I meant “that worked” mean it was outside but now its inside !!!

I have no clue why this is working

Pravin_Poudel · November 21, 2022, 2:44am

I found this in your code

github.com

Sean-Bradley/TeleportVR/blob/b076678b3571e20c53099fc2a5b369b91de626e5/src/client/demo1.ts#L91


      
          
          
const lefthand = new THREE.Mesh(
              new THREE.CylinderGeometry(0.05, 0.05, 0.4, 16, 1, true),
              new THREE.MeshBasicMaterial({
                  color: 0x00ff88,
                  wireframe: true,
              })
          )
          
          
const controllerGrip0 = renderer.xr.getControllerGrip(0)
          controllerGrip0.addEventListener('connected', (e: any) => {
              controllerGrip0.add(lefthand)
              teleportVR.add(0, controllerGrip0, e.data.gamepad)
          })
          
          
const righthand = new THREE.Mesh(
              new THREE.CylinderGeometry(0.05, 0.05, 0.4, 16, 1, true),
              new THREE.MeshBasicMaterial({
                  color: 0x00ff88,
                  wireframe: true,
              })

and wanted to add that event listener and only add controller model when it is connected

function buildControllers() {
    const controllerModelFactory = new XRControllerModelFactory()
    const controllers: Array<any> = []

    const rayGeometry = new BufferGeometry().setFromPoints([
        new THREE.Vector3(0, 0, 0),
        new THREE.Vector3(0, 0, -1),
    ])
    const line = new Line(
        rayGeometry,
        new THREE.LineBasicMaterial({
            color: 0x333300,
            linewidth: 5,
        })
    )
    line.scale.z = 0

    for (let i = 0; i < 2; i++) {
        const _controller = renderer.xr.getController(i)
        _controller.add(line.clone())
        _controller.name = 'controller' + (i + 1)
        _controller.userData.selectPressed = false
        _controller.userData.selectPressedPrev = false
        // add ray in it
        user.add(_controller)
        controllers.push(_controller)

        const grip = renderer.xr.getControllerGrip(i)
        grip.addEventListener('connected', (e: any) => {
            let grip1 = e.target
            grip1.add(controllerModelFactory.createControllerModel(grip1))
            user.add(grip1)
            console.log(user)
        })
    }
    return controllers
}

but when I put these inside the connected event, the controller model is not rendered !!

user is the group where I am putting camera inside it !!!

    user.add(camera)
    user.position.set(0.0, 0.0, 0.0)
    // user.rotation.copy(camera.rotation)
    scene.add(user)

i wanted to ask this question to get more idea if I am rendering it right or not !!

This is my code in animate function:


function animate(now: number) {
    let delta = clock.getDelta()
    delta = Math.max(delta, 0.1)
    world.step(delta)
    if (orbitControls.enabled) {
        orbitControls.update()
    }
    cannonDebugRenderer.update()
    TWEEN.update()
    KeyBoardHandler.keyUpdate(handlers, keys, delta * 1000)
    if (character) {
        character.position.copy(camera.position)
    }

    updateControllerAction()
    render(delta)
}

function render(delta) {
    renderer.render(scene, camera)
    labelRenderer.render(scene, camera)
}

and I have this for animation loop

renderer.setAnimationLoop(animate)

I am sorry, I am not sure about my code so i am asking

Pravin_Poudel · November 22, 2022, 3:50am

Hi,

this is my update

I added this ray in the controller and this target sprite at end of the ray !!

Pravin_Poudel · December 19, 2022, 7:13am

@jrjdavidson I fell into the same problem again !!

can you please help me with this !!

jrjdavidson · December 22, 2022, 8:26am

My only advice is what has already been said here before: make sure that your model is at a roughly metre scale( I.e.: 1 unit in your scene is 1m) and that your origin is where you want to start in VR mode. It just makes all the rest easier.

Pravin_Poudel · December 23, 2022, 1:42am

I resized the room with this

function normalizeSize(object, scale, needReposition, roomName: string | undefined = undefined) {
     let center = new THREE.Vector3()
     let size = new THREE.Vector3()

     let box = new THREE.Box3().setFromObject(object) //compute AABB
     box.getCenter(center)
     box.getSize(size)

      let maxSideDimension = Math.max(size.x, size.y, size.z)
      object.scale.multiplyScalar((1.0 * scale) / maxSideDimension)

        ;(rooms[roomName] as any).size.x = ((size.x * scale) / maxSideDimension) as number
        ;(rooms[roomName] as any).size.y = (size.y * scale) / maxSideDimension
        ;(rooms[roomName] as any).size.z = (size.z * scale) / maxSideDimension
        ;(rooms[roomName] as any).center.x = 0
        ;(rooms[roomName] as any).center.y = 0.0
        ;(rooms[roomName] as any).center.z = 0

    return object
}

and on gltf model load, after traversing is finished, i do this

     let object: any = gltf.scene
     object = normalizeSize(object, 3, false, 'livingRoom')
     scene.add(object)

jrjdavidson · December 24, 2022, 7:50am

If it was me, I would set the scale and origin up the scene in something like blender before importing to threejs

Topic		Replies	Views
WebXR camera position and rotation issue? Questions webxr	1	537	November 10, 2023
Finding the local camera.position in an XR session Questions camera , webxr	0	1097	February 10, 2022
Webxr camera start at different position than in normal view position Questions webxr	0	541	December 19, 2022
Not able to reset camera position in WebXR Questions camera , perspective-camera , webxr	3	1879	February 1, 2021
WebXR: camera's position not changing when using OrbitControls Questions webxr , orbitcontrols	7	3867	February 27, 2022

Webxr camera is not at position of perspective camera

Related topics