I am looking at getting video in my scene. I know that the only way to get a Youtube embed in the scene is using the CSS3DRenderer (is this still true?). But i want to check what the limitations are before I implement.
Does the css3D object act like a normal 3D object when it’s in my scene? So for example, if I place a plane playing a youtube video inside of a 3D room, then leave that room - will the walls/room block my view of the css3D object or will it just render over other things?
Would is be a big drain on performance to use both renderers?
No, that does not happen. The main purpose of CSS3DObject is to get the transformation in 3D space right. However, HTML/CSS based elements are decoupled from WebGL content. This becomes very clear in context of depth sorting. Meaning WebGL content comes first then HTML or vice versa.
You could also consider to save the video on your server and then use THREE.VideoTexture. In this case, you can treat the video as a texture and use it with a material for WebGL rendered objects. The following example demonstrates this approach:
The hope was to push a YT stream to a plane within a 3D space. I found this, so will study it and try to implement it mixed with the official youtube css3d example.
Failing that - will revert to using stored videos for the project.
The idea is to first render your css scene and then render the webgl scene with a hole, where your video would be. This works really well, but does have a few limitations:
The browser needs to support alpha for your canvas. Edge (not the new blink based one) and IE do not support that.
You will have to decide how to handle clicks and interactions with the css objects. You can make them clickable while also make the webgl scene interactable (as seen in the example @Mugen87 posted), but since youtube videos are embedded as iframes you can catch events through bubbleing. I found no solution to block events for css iframe objects when they are behind webl objects while keeping them interactable when visible. For normal html elements this is possible, since you can control the events a lot better when they are in your html document.
The CSS3D Renderer uses the power of Matrix3D. I recommend not to use any special css rules for your css scene element. I had problems with flexbox in combination with the css3d element on chrome and safari (firefox handled it perfectly well).
All in all: Totally doable, but has some edge cases to look out for
OK - this is going to be very helpful, thanks a lot.
I am willing to leave out Edge and IE if the result is possible. I understand the basic premise of what you’re saying and i’m sure reading through the codepen will fill in the blanks. Thanks again!
Hi @JohannesDeml - to what extent does both renderers lining up rely on running at 60fps? On my old iPhone7 - I was getting about 17fps. It appeared like both renderers were lagging behind each other. It created a kind of tearing at the edges of the iframe. I tried to take a screen recording of it happening and it doesn’t happen when i’m screen recording. (this makes me think it’s not tied to fps?) which Is why I ask my initial question.
@Heiddo Very good question. I can’t give you a definitive answer. I do notice a small delay in the css renderer as well on my iphone se. I had a lot more delay when I was using an underlying flexbox, so it might have to do with the complexity of your css structure? With my current setup I have no delay on firefox and chrome on my desktop and 1 or 2 frames delay on iOS, which is totally okay for my use case. I additionally set a background color for the css that is most of the times the color that would be rendered in the webgl, so the delay is even less visible and creates no visible artifacts at the edges.
If your question was more like "does this eat up so much performance, that it is only possible to render the scenes with 17 FPS, the clear answer would be no, not at all.
All that being said, I didn’t find a proper cause for the delay on iOS nor a solution to get rid of the delay entirely. I imagine it has to do with updating the css matrix in the correct order. If that would be the problem I fear there is not much we can do, since that’s something the browser has to handle.
No no, I understand this wasn’t the ‘cause’ of the my 17fps. Rather having such low fps made it obvious. And I find it extremely strange that starting a screen recording on iPhone gets rid of the issue!
Yeah it’s a real shame, I got beautiful results on desktop and android devices.
@Heiddo fyi, I hit another minor device specific problem with iPhone X and iPhone 11. For those two devices there is a small offset in the matrix, can’t quite tell why that happens
@JohannesDeml - something I noticed in my project regardless of device was that resizing a window mid load or rotating a phone screen post / mid load would (often) then offset the CSS layer by quite a bit.
Hopefully you figure it out. I’m going to test your error on an older iPhone when it turns itself on.
it doesn’t happen when i’m screen recording. (this makes me think it’s not tied to fps?) which Is why I ask my initial question.