Problem with interaction when merging WebGLRenderer & CSS3DRenderer

Hi,

I am facing an issue if I would add WebGLRenderer & CSS3DRenderer together. And if I can make the CSS to properly oculate the WebGL objects. Then I find I can not interact with CSS3DObjects and if I would set pointer event none to WebGL canvas, I see that I can interact with CSS3DObjects but I cannot interact with WebGLRenderer components such as OrbitControls, RayCaster etc. So basically I can see that I can only interact with either of one renderer. Can you please provide me a workaround?

I tried to make an example with two WebGL cubes and one HTML tag – they both react on mouse hover. The cubes use raycasting, the tag uses CSS. There is OrbitControls acting on the cubes and the tag.

Because this is my first time ever using CSS3DRenderer there is one glitch - the HTML tag is not shown at the beginning. If you rotate the scene a bit, it will show up.

https://codepen.io/boytchev/full/GRXEdEx

Hi @PavelBoytchev, please allow me some moment I will try to create a small example defining the structure I am using in codepen then sharing with you.

One more thing I noticed is that, in your example the html is not getting occulated by the 3d object.

image

Yes, this is so. I’m not experienced with CSS3DRenderer, so I only focused on having interaction of both WebGL and HTML elements.

But bro I can integrate it and I can get it working when it’s not properly occulated. But it doesn’t let me interact with when I try to make it perfect with occultation. For example, in your case if you increase the z-index of the WebGL canvas so I hope it will give proper occlusion but then we cannot interact with CSS3DObject. And if we set WebGL canvas pointer-events:none then, occultation and interaction with CSS3DObject both will work but OrbitControls and WebGL raycasting will not work. This is the problem.

Not sure if this is helpful:

From the Collection of examples from discourse.threejs.org

GradientsMaterialColor

1 Like

Hi @hofk Sorry for replying late. I was doing some experiments with all the possibilities. The sample you have provided me with, are lacking two things, 1) It does not have OrbitControls in it. If it has then it would fail to work due to WebGL canvas has pointer-event:none set. 2) I can interact with the textarea through the WebGL object. This is not expected behavior.

How is it possible to restrict clicking the CSS3DObject through the WebGLObject?

As you can see in the source code, the example is not mine. I only collect the examples. Maybe the author @trusktr can help?

Actually, the last question is about general situation, because with some experiment I have figured out closer result as per my expectation but that is having the problem of click reaches to the css3dobject through the webglobject. So, I asked. I am trying to provide an example in codepen for better understanding.

Hi @hofk, @PavelBoytchev, please check the example of the closest I can do as per my expectation but as you can see user can click the video through the cube which is not expected.

WebGL & CSS3D Interaction Demo

Can you guys please suggest me a workaround to avoid this?

What reason to have two controls for the same camera?

I would comment out these two lines:
controls = new OrbitControls(camera, renderer.domElement); - as you set pointer-events: none; for the canvas of webgl renderer
controls.update(); - not needed, as you don’t have controls (as you commented it out)
:thinking:

I can now understand the problem. Unfortunately, I don’t know a solution for the exact approach that you use. Maybe someone experienced with mixing renderers might help.

If the goal is to show a video on the surface of a 3D mesh, this can be done with video textures, so there is no need to use second renderer. However, there is a limitation – you must provide a link to the video file (i.e. not iframe with URL to a web page). Would this work for your project?

Yeah right, the control for the for WebGLRenderer is not needed. Actually, I was experimenting with different approaches. So kept like this. But I hope this doesn’t make any difference as of current situation of the code.

Actually, we need to have iframe there cause the content is not limited to just video, rather it could be any web content, even a webpage can be there.

There is no solution available for what I am looking for, correct? So should I report this issue as bug in GIthub repository of Threejs?

I would not consider it as a bug, as it is not some existing (or required) functionality that is broken. CSS and WebGL are two incompatible worlds and the CSS renderer tries to make the CSS just a little bit more WebGL-ish.

So, what you want to do it more like a feature request, not a bug.

However, there might be a solution out there, that I’m not aware of – a solution with the appropriate level of magic and trickery that mixes smoothly CSS and WebGL.

Actually, I think in earlier version, it was working how I am looking for now as I can remember, but in current version it does not work like that. Also, finding all the previous example in which claims it to be working is broken adding more difficulties in it.

Not sure what you are doing different, but maybe try the suggestions from this topic:
CSS3DREDNERER object - Questions - three.js forum (threejs.org)
I get occlusion and orbit controls working (mostly) as expected (using v1.23, but I don’t think that is the problem)

2 Likes

Hi @awonnink, thank for trying to help me. Sorry for late reply, I was trying to recreate a simple scene based on the Threeml Earth-Moon example I have found from the thread you have shared. I still see some differences; means I cannot exactly recreate what is done in Threeml.

This is what I have derived from the Threeml Earth-Moon example. But I have seen in Threeml example container’s pointer event changes based on mouse movement, but in my case, it’s not working like that causing difference in behavior, I think. Can you please tell me what I am missing?

A Pen by Smithangshu Ghosh (codepen.io)