Active Html in VR

Using Css2d/Css3d interactive Html pages can be used in 3D scenes in the browser. However these components use a trick by putting the html part on top of a 3d scene. It is not affected by light/shadow and, perhaps more important, it doesn’t work in VR.

What needs to be done to allow a browser to display panels with (interactive) webpages in VR?
Do you know of any group of people working on a specification for this, like at the W3C?

1 Like

This specific issue has been annoying me for almost two years.

Short answer :
Impossible.

Long answer :
If you look at how the GUI is rendered in this example, you will see that @mrdoob created an undocumented HTMLMesh class, which you can use to render a dom element on a texture which is then rendered on a plane mesh in the scene. However this is a hack, and complex CSS will break.

Another option is to use a GUI that does not depend on HTML/CSS. There is three-mesh-ui (disclaimer: I’m the creator of this lib), or troika-three-text.

You’re distressed by my answer and it’s understandable, however I’m glad to announce that the future is bright: There is a Web API that will support this in the future (probably). It will be called DOM Layers and it will be a subset of XRLayers, see the W3C repo here if you want to follow the development of the specs. From what I understand it will land in mid-2022.

3 Likes

If you are using webxr to create VR Button to start a three VR session it’s possible…

Very interesting, and I am glad more people are missing this! It seems obvious to me that in a ‘Metaverse’ of whatever you want to call it, we want browser functionality just to extend to the extra dimension.

I looked at the W3C link, but there is no mention of .DOM layer… Do you perhaps have more info specific on this subset?

Is this just to overcome the ‘VR NOT SUPPORTED’ message and does it use Css3DObject, or am I missing the clue?

Maybe it’s me that’s confused, yes using css3d renderer and xr Session, using googleARservices on AR enabled devices this seems to work fine, unless I miss understood the question? You can see an example at http://xenzvr.com collecting the key from inside the e room E and entering the ‘secret’ room

I should have mentioned that my question is about immersive VR (and AR). I am told that the css3drendering doesn’t work there and I assumed this is because these devices can only use the WebGL layer.

Hey there,

is there an update on this, currently have the same problem?
It is indeed mid 22 but I do not see progress on this.

I would also be interested to hear if there has been any progress on this that anyone could share? Thanks for the WebXR layer links @felixmariotto, I have been reading up on those. I also came across this WebXR DOM Overlays Module that could be useful in the future.
I also just learnt the hard way by putting together a project using Css3d (rendering a iframe with input support) only to learn when launching into “Immersive XR” on my phone or Meta Quest 2 than none of the CSS3d gets rendered.

This seems like a vital element for the future of WebXR / Metaverse development and growth.

2 Likes

Maybe the most preferred implementation would be if we could apply active Html as a texture, thereby supporting more than flat planes.

@JDihlmann @Jagged I’m also watching for a sign of life for DOM Layers but I haven’t seen anything new sorry. @Jagged From what I understand DOM Overlay will never work on headmount systems, it’s intended for handheld systems (see discussion here). @awonnink WebXR Layers already support more than flat planes, so hopefully when (if?) they add DOM Layers we should be able to make curved surfaces right away.

3 Likes

Such as converting a html element to a canvas texture? Theres html2canvas and html2image libraries which can do this quite rapidly but of course never going to be real time

1 Like

This will be a huge improvement, but maybe not the ‘first principles’ choice of implementation. As a technology, at least we should be able to implement real world behavior of a browser in VR/AR. For me, this includes flat and curved screen browsing. But the proposed solution will probably allready fail when trying to simulate foldable-, and more exotic future screens.
And we mis out on the chance to make any model’s surface browsable …

2 Likes

@JDihlmann @Jagged @awonnink :tada: :tada:

2 Likes

Fantastic news. I was literally reading this yesterday as well and feeding back to my team. I should of commented something here as well to let you know. I saw you added a “bug” to the project asking about any updates and linking back to this discussion.
Awesome to see this is still moving forward and that progress is being made :partying_face:

1 Like