A couple of years ago, while working on a VR demo for Splunk, I became obsessed with the idea of using the limitless screen real estate of VR for complex business applications. I wrote a couple of Medium articles on the topic (starting with Enterprise VR: Will it happen?) but it wasn’t until 2024 that I got serious about implementing and testing these concepts for practical use cases (using a Quest 3 as the best bang-for-the-buck device available at the time).
The approach I took for interactive 2D interfaces is to create a three.js-based framework for rendering into canvas-based textures which emulates something akin to React on top of something akin to (but much much leaner than) DOM. My reference application at this time is the Veer Index, a companion app for a Sci-Fi novel.
The problem that almost made me give up on my approach was the blurriness of fonts in WebVR on Quest 3 and Galaxy XR. SDF font rendering (I tried troika) seemed to be the obvious solution. But texts rendered with troika created jarring shimmer effects at a distance and weren’t actually much sharper. Also, moving from using the Canvas API to 100% shader-based rendering of visually rich 2D content would have meant a lot of complications.
The key to the solution was observing that the Virtual Desktop, which is nothing but a streamed bitmap, had reasonably crisp font rendering. It turned out that when using three’s WebXR, the UI will not use the full (native) resolution of the device, at least not for Quest 3.
I had to manually “upgrade” by calling renderer.setFramebufferScaleFactor(FRAMEBUFFER_SCALE);
XRWebGLLayer.getNativeFramebufferScaleFactor() reported a value of 1, which turned out to be wrong. I empirically arrived at a value of at least FRAMEBUFFER_SCALE=1.35 to get the crispness of the fonts I expected. I thought that it would be good to share this information here in case anybody else runs into this issue. After almost giving up on spatial web applications, this insight has been a turning point for me. I’d be glad to answer any other questions around emulating 2D User Interfaces in a three.js scene.
Happy hacking!
Horst