TLDR: after playing a heavy holographic video file (MRCS ) in an 8th Wall / THREE.js app, shader transparency stops working.
Hi all, I’m currently working on an AR project using 8th Wall.
The project contains 2 holographic videos ( A-Frame: MRCS Hologram | 8th Wall | 8th Wall for an example ), as well as a few models of products.
The flow of the app is:
user scans an image target on the wall
an animation plays in order to reveal the corresponding holographic video or product
A quick bit of background on how the holographic video works: geometry is contained in a bunch of .bin files, which each correspond to a few seconds of the video. In parallel, an mp4 video is streamed with the texture data.
From a code perspective, this is all handled by some Microsoft proprietary code that I got a minified copy of with no possibility to access the source. It’s hooking directly into the THREE.js renderer and doing a lot of low level WebGL stuff.
A few days ago, I noticed that the particles I’m using rendered differently after playing a holographic video. I was using a texture to give them circular shapes, and after playing the video the circular shape seemed to have gone and they rendered as squares. I thought my phone was probably running out of memory, so I switched from using an alpha map to creating the circle inside the fragment shader, which solved the issue.
Then I ran into an issue with some other textures I was using to simulate quick camera flashes: they stopped rendering as transparent after the video played, instead the transparent pixels rendered black.
Again, I converted these images to shaders.
Unfortunately, that hasn’t solved the issue.
The shaders render as expected if I go through one of the animations that doesn’t include a holographic video.
But after I play a video, it looks like transparency stops working. I can set gl_FragColor.a = 0. at the end of the fragment shader, to no effect. Yet modifying the rgb values does work, so does calling discard .
The additive blending also stops working.
Unfortunately I can’t post a codepen to reproduce the issue, as that would involve sharing the holographic videos.
Any ideas of what I could do to work around this? Unfortunately, using discard instead of setting gl_FragColor.a won’t work, as these shaders require the opacity to follow a gradient instead of a sharp cutoff.