Using Css2d/Css3d interactive Html pages can be used in 3D scenes in the browser. However these components use a trick by putting the html part on top of a 3d scene. It is not affected by light/shadow and, perhaps more important, it doesn’t work in VR.
What needs to be done to allow a browser to display panels with (interactive) webpages in VR?
Do you know of any group of people working on a specification for this, like at the W3C?
This specific issue has been annoying me for almost two years.
Short answer :
Impossible.
Long answer :
If you look at how the GUI is rendered in this example, you will see that @mrdoob created an undocumented HTMLMesh class, which you can use to render a dom element on a texture which is then rendered on a plane mesh in the scene. However this is a hack, and complex CSS will break.
Another option is to use a GUI that does not depend on HTML/CSS. There is three-mesh-ui (disclaimer: I’m the creator of this lib), or troika-three-text.
You’re distressed by my answer and it’s understandable, however I’m glad to announce that the future is bright: There is a Web API that will support this in the future (probably). It will be called DOM Layers and it will be a subset of XRLayers, see the W3C repo here if you want to follow the development of the specs. From what I understand it will land in mid-2022.
If you are using webxr to create VR Button to start a three VR session it’s possible…
Very interesting, and I am glad more people are missing this! It seems obvious to me that in a ‘Metaverse’ of whatever you want to call it, we want browser functionality just to extend to the extra dimension.
I looked at the W3C link, but there is no mention of .DOM layer… Do you perhaps have more info specific on this subset?
Is this just to overcome the ‘VR NOT SUPPORTED’ message and does it use Css3DObject, or am I missing the clue?
Maybe it’s me that’s confused, yes using css3d renderer and xr Session, using googleARservices on AR enabled devices this seems to work fine, unless I miss understood the question? You can see an example at http://xenzvr.com collecting the key from inside the e room E and entering the ‘secret’ room
I should have mentioned that my question is about immersive VR (and AR). I am told that the css3drendering doesn’t work there and I assumed this is because these devices can only use the WebGL layer.
Hey there,
is there an update on this, currently have the same problem?
It is indeed mid 22 but I do not see progress on this.
I would also be interested to hear if there has been any progress on this that anyone could share? Thanks for the WebXR layer links @felixmariotto, I have been reading up on those. I also came across this WebXR DOM Overlays Module that could be useful in the future.
I also just learnt the hard way by putting together a project using Css3d (rendering a iframe with input support) only to learn when launching into “Immersive XR” on my phone or Meta Quest 2 than none of the CSS3d gets rendered.
This seems like a vital element for the future of WebXR / Metaverse development and growth.
Maybe the most preferred implementation would be if we could apply active Html as a texture, thereby supporting more than flat planes.
@JDihlmann @Jagged I’m also watching for a sign of life for DOM Layers but I haven’t seen anything new sorry. @Jagged From what I understand DOM Overlay will never work on headmount systems, it’s intended for handheld systems (see discussion here). @awonnink WebXR Layers already support more than flat planes, so hopefully when (if?) they add DOM Layers we should be able to make curved surfaces right away.
Such as converting a html element to a canvas texture? Theres html2canvas and html2image libraries which can do this quite rapidly but of course never going to be real time
This will be a huge improvement, but maybe not the ‘first principles’ choice of implementation. As a technology, at least we should be able to implement real world behavior of a browser in VR/AR. For me, this includes flat and curved screen browsing. But the proposed solution will probably allready fail when trying to simulate foldable-, and more exotic future screens.
And we mis out on the chance to make any model’s surface browsable …
Fantastic news. I was literally reading this yesterday as well and feeding back to my team. I should of commented something here as well to let you know. I saw you added a “bug” to the project asking about any updates and linking back to this discussion.
Awesome to see this is still moving forward and that progress is being made 
Kia ora koutou, just stumbled across this thread, I’m real keen on this capability too, it’s great to see other are eagerly anticipating this too.
Not sure how soon this is going to land, so I’ve been trying to update the threejs HTMLMesh example so it works for my use cases ( e.g. fires pointer events, wrapping text) in the mean time.
Hello, how did you manage to do this (display a web page)?
I need similar solution, I’m using CSS3DRenderer together with WebGLRenderer, but VRButton is not supporting CSS3DRenderer, so html is not displayed in VR.
Any help would be great!
@valvkvc The example above used DeviceOrientationControls so not technically VR, the API has since been updated and these controls removed from three r134
Afaik it was not possible to do this using VRButton at the time of this thread as @felixmariotto and @awonnink outlined above, there were however some other libraries that were in progress at the time to try to achieve this too Ion 3D Engine showcase - VR User Interface (UI)
Hi,
some time ago I wrote a code that renders HTML elements to a canvas - I use this for a project to simply generate content for textures.
I have stored the code there:
Basically, I get the position, size, background color, text color + font, corner rounding of the elements; and images - and render that into a canvas element.
Classic input elements are also drawn. However, there is no interaction with these.
