Active Html in VR

Using Css2d/Css3d interactive Html pages can be used in 3D scenes in the browser. However these components use a trick by putting the html part on top of a 3d scene. It is not affected by light/shadow and, perhaps more important, it doesn’t work in VR.

What needs to be done to allow a browser to display panels with (interactive) webpages in VR?
Do you know of any group of people working on a specification for this, like at the W3C?

This specific issue has been annoying me for almost two years.

Short answer :
Impossible.

Long answer :
If you look at how the GUI is rendered in this example, you will see that @mrdoob created an undocumented HTMLMesh class, which you can use to render a dom element on a texture which is then rendered on a plane mesh in the scene. However this is a hack, and complex CSS will break.

Another option is to use a GUI that does not depend on HTML/CSS. There is three-mesh-ui (disclaimer: I’m the creator of this lib), or troika-three-text.

You’re distressed by my answer and it’s understandable, however I’m glad to announce that the future is bright: There is a Web API that will support this in the future (probably). It will be called DOM Layers and it will be a subset of XRLayers, see the W3C repo here if you want to follow the development of the specs. From what I understand it will land in mid-2022.

2 Likes

If you are using webxr to create VR Button to start a three VR session it’s possible…

Very interesting, and I am glad more people are missing this! It seems obvious to me that in a ‘Metaverse’ of whatever you want to call it, we want browser functionality just to extend to the extra dimension.

I looked at the W3C link, but there is no mention of .DOM layer… Do you perhaps have more info specific on this subset?

Is this just to overcome the ‘VR NOT SUPPORTED’ message and does it use Css3DObject, or am I missing the clue?

Maybe it’s me that’s confused, yes using css3d renderer and xr Session, using googleARservices on AR enabled devices this seems to work fine, unless I miss understood the question? You can see an example at http://xenzvr.com collecting the key from inside the e room E and entering the ‘secret’ room

I should have mentioned that my question is about immersive VR (and AR). I am told that the css3drendering doesn’t work there and I assumed this is because these devices can only use the WebGL layer.