Object sorting ignores Scene.overrideMaterial

Hi!

I’m wokring on a post processing effect that needs to render the scene with a custom material to work. Scene.overrideMaterial is perfect for this. The issue is, this custom material needs to be transparent for the effect to work properly. I noticed that the algorithm that sorts objects before each render seems to ignore Scene.overrideMaterial, so even though the overrideMaterial is transparent, objects that have opaque materials still get in the opaque render list.

I created a small CodeSandbox to illustrate: https://codesandbox.io/p/sandbox/transparent-override-material-xltngw

As you can see, if only overrideMaterial is set to transparent, the cubes are rendered transparent, but they still occlude each other, because they are ordered front to back.

Is this intentional? If yes, can you explain why? If not, I would be happy to try creating a bug fix for this.

1 Like

It does sound like a legitimate issue, but I don’t think it’s fixable just by having .transparent respected on the overrideMaterial.. as that might break existing code.

Can you swap out:
WebGLRenderer.setOpaqueSort
and/or
WebGLRenderer.setTransparentSort

during your pass?
At least you would get consistent sorting in both passes, even though all “opaque” materials would render first.. then .transparent=true materials… but you would still be able to sort the opaque back to front.
I know you will still have bucketing into 2 lists..

Maybe an added flag on renderer to force or disable transparent/opaque bucketing, and thus render everything in one list with your own custom sortFunction?

… even though all “opaque” materials would render first.. then .transparent=true materials…

Unfortunately this wouldn’t be a robust solution. If I have both opaque and transparent materials in my scene (which I have), then having two render lists would produce improper results in certain cases.

I don’t think it’s fixable just by having .transparent respected on the overrideMaterial.. as that might break existing code.

How could that break existing code? If the renderer is not meant to behave like this, then this is should be considered a bug. If some code out there in the wild is building on this behaviour - which would be kind of weird -, then yes, it could break, and that’s unfortunate, but I don’t think that should dictate whether a bug is fixed or not. So I think the question really is, should this be considered a bug or not?

According to the docs, Scene.overrideMaterial

Forces everything in the scene to be rendered with the defined material.

I think this is not satisfied by the current implementation, as the output is different than if I traverse the scene and replace every material of every object manually.

1 Like

Just thinking aloud: not sure about intention, but maybe overriding happens at the end of the processing pipeline, i.e. after splitting transparent from non-transparent and their sorting. Also, somebody might want to keep the original sorting, but just use a different appearance of the material, so for this person the current behaviour is the expected one.

1 Like

@ferenczyg I agree the docs are misleading… But the case that immediately comes to mind that may exploit this behavior, is shadowmapping. As it stands, a single external shadowmapping shader could still render both the opaque and transparent lists via overridematerial.. vs requiring 2 different materials (one with .transparent=true and one with .transparent=false) to achieve proper sorting for shadow mapping. I looked through the threejs library source, and also found “transmission” may exploit this behavior as well.

I’m not implying that your observation is invalid, I’m just saying that there is a lot more inertia toward changing existing behavior, than adding extra conditions to control it, and/or clarifying the docs to explain the nuances.

Perhaps 2 new .overrideMaterials, the existing one for simultaneous opaque/transparent, and 2 separate ones for opaque/transparent independently… Kinda like how .matrixAutoUpdate was split out with .matrixWorldAutoUpdate, or the buffer clear .autoClear were split out into .autoClearDepth, .autoClearColor to give finer grained control.

This approach means existing code (I mean stuff people have never even made public but still uses this behavior) doesn’t break, but you will be free to use the additional methods.

Regardless, This definitely sounds worth filing an issue on the github repo to elevate the conversation to the folks that will actually hash out how to address it.

Please post back when you file the issue! I’d be curious to see what the feedback is, and I’d like to support it getting addressed. :slight_smile: (It may be that I’m completely wrong, and they just decide to change the existing behavior, as you described.)

edit: a quick glance at the open issues has:

which may be related.

2 Likes

Thanks for the examples, these really help putting the “may break existing behaviour” argument into context. I would be just as happy with the “adding extra overrideMaterial options” solution as long as the current implementation has some actual purpose, or at least some real life practical use, which, as you mentioned, it seems to have.

I will go through the issues you linked and try to find others with similar topics. If I don’t find anything I could piggyback on, I will open a new one and link it here.

1 Like

I went through the existing issues and I don’t think they sufficiently cover this topic, so I opened a new one: Object sorting ignores Scene.overrideMaterial · Issue #32104 · mrdoob/three.js · GitHub

1 Like