Using Css2d/Css3d interactive Html pages can be used in 3D scenes in the browser. However these components use a trick by putting the html part on top of a 3d scene. It is not affected by light/shadow and, perhaps more important, it doesn’t work in VR.
What needs to be done to allow a browser to display panels with (interactive) webpages in VR?
Do you know of any group of people working on a specification for this, like at the W3C?
This specific issue has been annoying me for almost two years.
Short answer :
Long answer :
If you look at how the GUI is rendered in this example, you will see that @mrdoob created an undocumented HTMLMesh class, which you can use to render a dom element on a texture which is then rendered on a plane mesh in the scene. However this is a hack, and complex CSS will break.
Another option is to use a GUI that does not depend on HTML/CSS. There is three-mesh-ui (disclaimer: I’m the creator of this lib), or troika-three-text.
You’re distressed by my answer and it’s understandable, however I’m glad to announce that the future is bright: There is a Web API that will support this in the future (probably). It will be called DOM Layers and it will be a subset of XRLayers, see the W3C repo here if you want to follow the development of the specs. From what I understand it will land in mid-2022.
Very interesting, and I am glad more people are missing this! It seems obvious to me that in a ‘Metaverse’ of whatever you want to call it, we want browser functionality just to extend to the extra dimension.
I looked at the W3C link, but there is no mention of .DOM layer… Do you perhaps have more info specific on this subset?
Maybe it’s me that’s confused, yes using css3d renderer and xr Session, using googleARservices on AR enabled devices this seems to work fine, unless I miss understood the question? You can see an example at http://xenzvr.com collecting the key from inside the e room E and entering the ‘secret’ room
I would also be interested to hear if there has been any progress on this that anyone could share? Thanks for the WebXR layer links @felixmariotto, I have been reading up on those. I also came across this WebXR DOM Overlays Module that could be useful in the future.
I also just learnt the hard way by putting together a project using Css3d (rendering a iframe with input support) only to learn when launching into “Immersive XR” on my phone or Meta Quest 2 than none of the CSS3d gets rendered.
This seems like a vital element for the future of WebXR / Metaverse development and growth.
@JDihlmann@Jagged I’m also watching for a sign of life for DOM Layers but I haven’t seen anything new sorry. @Jagged From what I understand DOM Overlay will never work on headmount systems, it’s intended for handheld systems (see discussion here). @awonnink WebXR Layers already support more than flat planes, so hopefully when (if?) they add DOM Layers we should be able to make curved surfaces right away.
This will be a huge improvement, but maybe not the ‘first principles’ choice of implementation. As a technology, at least we should be able to implement real world behavior of a browser in VR/AR. For me, this includes flat and curved screen browsing. But the proposed solution will probably allready fail when trying to simulate foldable-, and more exotic future screens.
And we mis out on the chance to make any model’s surface browsable …
Fantastic news. I was literally reading this yesterday as well and feeding back to my team. I should of commented something here as well to let you know. I saw you added a “bug” to the project asking about any updates and linking back to this discussion.
Awesome to see this is still moving forward and that progress is being made