Occlusion Between CSS3D Renderer and WebGL Renderer

[trusktr]

Nice you got it! Yep, it just cuts a hole. Indeed, if the post processing pass is not aware of the hole and applies colors over the hole, then it will mess with the effect. There’s ways around that, f.e. render the NoBlending object on top of the post-processed content, but the to the effect will not apply to the HTML/CSS content. So there are difficulties to overcome when trying to mix the CSS layer with the WebGL layer.

If your HTML content is using a limited set of CSS properties, then what you can do is render the HTML to a texture first, the use that texture on a Mesh with PlaneGeometry, and then the content will be fully interactive with lighting, transparency, and post processing effects.

To do that,

(1) first put the HTML inside of a <foreignObject> element inside of a <svg> element, like so:

- SVG: Scalable Vector Graphics | MDN

once you have that, then

(2) convert the SVG to pixels. Clues are here:

ImageBitmap from SVG - Jagged Text for some Font Sizes

^ You might try rendering the SVG to 2D canvas directly.

More SVG-to-canvas considerations here:

Let's solve pixelated SVG texture rendering in Three.js

Is it solvable? Before starting, I’ve seen the other threads: using SVGLoader is not an ideal final solution for rendering absolutely any SVG content because SVGLoader cannot draw all SVG features, and features it supports often come out glitched and distorted. I’ve also tried changing mipmap modes, but that seems unrelated to the following. The only currently-available approach that I know of, for 100% SVG support, is to render an SVG image into a texture. However, as many of you reading know…

Finally, once you have the SVG rendered as pixels,

(3) Use a THREE.CanvasTexture to use the canvas that contains the SVG pixels as a texture, and use the texture on your material .map property.

Advantage of SVG-to-canvas:

• the result lives inside of the WebGL rendering, not making a hole in the rendering, so it looks like a native object inside the 3D scene with complete lighting and shading.

Disadvantage of SVG-to-canvas:

• if you want to use CSS animations, the hole approach will be fast but more visually limited with respect to WebGL effects, while the CanvasTexture approach will be slow because every frame you have to update the canvas with the new pixels, and to set canvasTexture.needsUpdate=true, which means every frame Three.js needs to upload the texture to the GPU again.

So it all depends on the use case, and what the limits are for various visual possibilities. You may animate a CanvasTexture so long as it fits within your budget (draw less things then you can spend more time on a simpler scene with CSS animations piped through canvas, or pipe only initial CSS content once without animation and keep it static (also miss out on certain CSS properties) then draw a more other things)