Rendering to a 2d canvas while XR is enabled

Questions
1

Hey all!

I am trying (trying) to figure out how to continue to render to (a/the) desktop canvas while webXR is enabled. Animation loops, etc all seem to stop rendering and either freeze or turn to black while WebXR is enabled.

Have a couple of use cases for this:

  1. Continuing to show a “Desktop Preview” in browser while WebXR is enabled - for spectators, streaming, etc
  2. Rendering to a video feed for teleconferencing (my actual application) - currently I have a webrtc client set up and configured, but I don’t have the ability for those video clients to be able to see anything but me wandering around in a VR headset - would like to be able to render a third camera into the scene such that I can reposition it and have it broadcast my VR avatar to the other parties.

Anyone have any info on what loops, etc should be used to try and render to the browser while webxr is enabled? Is that currently possible in the wide world of webxr?

2

I think I would try to adapt this WeXR sample to three.js:

https://immersive-web.github.io/webxr-samples/spectator-mode.html

The important aspect is to disable WebXR if you render to the default or a custom framebuffer (render target) and enable if again after the render call.

1 Like
3

That’s awesome, and exactly what I needed. Thanks for your help, gonna hop back in on trying to adapt it this weekend.

4

For future spelunkers who might discover this thread, here’s the trick I used to get in-VR previews working in r127 in my animation loop:

    // Render the scene in WebXR
    renderer.render(this.scene, this.camera);

    // Re-Render the scene, but this time to the canvas (don't do this on Mobile!)
    if (renderer.xr.isPresenting) {
        renderer.xr.enabled = false;
        let oldFramebuffer = renderer._framebuffer;
        renderer.state.bindXRFramebuffer( null );
        //renderer.setRenderTarget( renderer.getRenderTarget() ); // Hack #15830 - Unneeded?
        renderer.render(scene, camera);
        renderer.xr.enabled = true;
        renderer.state.bindXRFramebuffer(oldFramebuffer);
    }

It would be nice if some of the examples could show this off (or if three.js could integrate it as a more standard feature).