Progressive lightMap as Floor Shadow Catcher

So react three fiber has turned the progressive shadow example into an excellent shadow catcher

Faucets, select highlight - CodeSandbox
Baking soft shadows - CodeSandbox

Was wondering how i would go about doing the same in three js ? or is this even a valid application ?
and is there any docs available for this class ?

the scene contents will be:

  • a ground plane which receives shadow and acts as the target for progressive lightmap
  • a Gltf model which uses meshStandardMaterial for everything , it will cast shadows but not receive them
  • hdri for environment
  • array of directional light on top of everything spread around to cast ambient occlusion like shadows

update: after turning down the scale and position of the light jitter, the shadows seems more polished

@zalo hi !

using ā€˜addObjectsToLightMapā€™ iā€™m adding the lights , floor and the meshes in the ProgressiveLightMap instance

i donā€™t want the meshes to participate in receiving shadows , disabling
castShadow & receiveShadow has no effect

any suggestions on achieving this ?

maybe this helps,

this was the topic i opened about it: How can i "bake" a shadow onto a planeGeometry? very informative tips by @gkjohnson that helped a lot

and the new lightmap class, drei/AccumulativeShadows.tsx at 9baf3d86168f14d0a3a4754a037dc5ced00d7fe3 Ā· pmndrs/drei Ā· GitHub

  • removed potpack
  • removed uv2, changed shader into diffuse
  • new uvmat shader discards all pixels
  • removed attach logic, it takes a plane mesh and the regular scene
  • will finally ā€œbakeā€ results when frames run out

it has new methods:

  • clear() clears buffers and collects lights and meshes
  • configure(object) set the plane
  • prepare() sets all lights to 0, all meshes to a new uvmat
  • finish() sets back all lights and meshes to normal

you use it like this:

  • configure(yourPlane) once
  • clear() once the scene has loaded fully

now the loop:

// Switch rando lights on
lights.visible = true
// Shut down all scene lights, all meshes get discard-uv-mat
plm.prepare()

// Jiggle rando lights
lights.children.forEach((light) => light.update())
// Let the LM render
plm.update(camera, 100)

// Switch lights off
lights.visible = false
// Restore scene
plm.finish()
1 Like

converting this to class is out of my scope right now , canā€™t switch to react , will come back to this another day , big thanks !

hope @zalo can figure out an simpler alternative in the future since the potential of this as a shadow catcher is immense

but honestly i donā€™t know ā€¦ oop doesnā€™t seem to lend to something like that ā€¦ a component makes sharing naturally easy, hence it grows into an eco system, a class based approach will just never have that. :confused:

Three-gpu-pathtracer uses classes to define a similar kind of behavior. And at some point I hope weā€™ll get lightmap and ao map baking into the project, as well, which is just the path-traced version of this work. You may not be as in love with the OOP patterns for implementing something like but itā€™s very doable and very usable.

And in terms of share-ability I strongly disagree. Classes are just as easily shared and reused across projects and are arguably more shareable since vanilla (or three.js-only) donā€™t require users to buy into react in order to use or contribute to them. React implementations can always be made on top of vanilla implementations - the opposite direction is not as simple in my experience.

7 Likes

your library, threejs, the dom, are perfect examples of where classes are useful. but this is not what i am referring to. the lack of re-usability rather.

a matter of opinions, not so important right now. :slightly_smiling_face:

1 Like

Thereā€™s no doubt that good classes can be more complex to write and can easily hide interconnected environment spaghetti dependencies. Those kind of dependencies can be required in functional architectures, as well, but itā€™s definitely more generally frowned upon there. Thatā€™s all to say its easy to write bad code anywhere but itā€™s true something like a component or functional architecture can more easily promote good practices.

Iā€™m only suggesting here that the effort to make a good architecture with vanilla here is worth it since it opens up the user and contributor base and to say itā€™s not possible or that OOP intrinsically runs counter to reusable architectures feels disingenuous. This is project might be something Iā€™d be interested in using but given the way I choose to write my code I cannot and Iā€™d be more likely to write my own competing solution instead of improving the one that everyone can benefit from. I see the value and desirability of the component architecture and understand why people use it but to me React is ā€œthe entire jungleā€ here.

1 Like

i am looking at the threejs/jsm example in question, or most jsm examples for that matter, and what i see are classes that make the environment conform to their needs.

if i can help with the shadow catcher, turning it into something more generic, gladly. otherwise, always enjoy to discuss with you!

Yes I agree that some of the three.js examples are not always all that elegant :sweat_smile: But I think this still speaks to the history of them which has classically been legitimate examples vs designed, reusable code. And I agree that making streamlined patterns for classes is more difficult because theyā€™re more open and donā€™t impose any existing patterns. My only suggestion is that I think itā€™s worth the effort if only for a more expansive userbase. Itā€™s one of the reasons I try to use as few dependencies as possible in my projects. But as you mention these are just differing perspectives and opinions!

otherwise, always enjoy to discuss with you!

Likewise! :grin:

3 Likes

Another attempt at this

changes:

  • Not adding the monkey and sphere into the potpack part. (doing a simple name to check skip these meshes)
  • Shadow calculation is done on button click instead of in the render loop.
  • Clear is done by disposing the renderTargets.

Rest of the code is the same

@SeeOn in your attempt you tried to directly convert the r3f shadows to vanilla right , have you tried this approach of stripping the original lightmap file ?

The shader stuff + code logic is overwhelming but iā€™ll keep trying

3 Likes

looking good.

the potpack stuff, i have completely removed it in my version, it is not needed for anything that i could see. after clearing that out the code became a lot more manageable.

i would still recommend comparing notes against the r3f thing now that you have it working. i changed the shaders a lot from the original. especially the colorBlend feature looks really nice. also the alpha logic could be useful, i catch only the shadows and the plane is transparent, that allows it to be more flexible as i donā€™t need to match the ground.

1 Like

Hey! Looking Great. And to answer your question, I have not gone in this route ā€¦ I was going to, but then I have to switch to some other stuff as per my work requirement.

Your result look promising. Looking forward to trying it out.

1 Like

Another update

changes

  • potpack removed
  • uv2 not required if lightMaptexture is not applied on material.lightMap
  • uvMat replaced with targetMat from r3f AccumulativeShadows
  • random lights and the shadow catching plane are made in the class itself
  • uv edge blurring logic removed

tried to do the r3f logic where there are NO extra Scenes but failed with some webGL error ,
and also tried to use the SoftShadowMaterial from r3f which just stayed fully transparent
will look at it these again

sandbox preview

3 Likes

the softshadowmaterial accumulates alpha, you use it like this:

update: (frames = 100) => {
  // Adapt the opacity-blend ratio to the number of frames
  const material = gPlane.current.material
  material.opacity = Math.min(opacity, material.opacity + opacity / frames)
  material.alphaTest = Math.min(alphaTest, material.alphaTest + alphaTest / frames)

  // Switch accumulative lights on
  gLights.current.visible = true
  // Collect scene lights and meshes
  plm.prepare()

  // Update the lightmap and the accumulative lights
  for (let i = 0; i < frames; i++) {
    api.lights.forEach((light) => light.update())
    plm.update(camera, frames)
  }
  // Switch lights off
  gLights.current.visible = false
  // Restore lights and meshes
  plm.finish()
},

opacity is most likely always 1
alphaTest removes areas the shadow didnā€™t touch, in drei defaults to 0.75 which is safe, looks better when higher
frames is the amount of frames that it accumulates, 100 is best imo. it divides opacity and alphaTest by frames.

the reason it accumulates like that is to make it less obvious that the plane is opaque at first, it fades in as it accumulates, that way you never really see the plane, only the shadow.

PS.

recently i worked on another drei primitive for caustics together with @N8Three and solved the same issue (making only caustics show on an otherwise transparent plane) in a completely different (but better) way, using blend modes. maybe you want to try it, it removes all the pesky alpha stuff. drei/Caustics.tsx at faceb40911e5826a65ea9e39fd25f2ba319b794d Ā· pmndrs/drei Ā· GitHub

          <causticsProjectionMaterial
            transparent
            ...
            blending={THREE.CustomBlending}
            blendSrc={THREE.OneFactor}
            blendDst={THREE.SrcAlphaFactor}
            depthWrite={false}
          />

i wanted to update accshadows with that as well but didnā€™t get to it yet. basically you just turn black areas and white areas into an alphafactor, add contrast and you can dial in something similar to alphaTest. so easy, but didnā€™t think of that back then.

since caustics are white and shadows black im guessing you would just need OneMinusFactor and it should work.

1 Like

the blending method if works will be amazing :crossed_fingers:

using the same options from caustics , the resulting image got brighter/overexposed ,

then i tried most of some other options

the one looking closest to the correct result was when i used

 new MeshStandardMaterial({
        color: 0xffffff,
        name: "shadow_catcher_mat",
        map: this.progressiveLightMap2.texture,
        transparent: true,
        blending: CustomBlending,
        blendEquation: MinEquation,

      })

but this result seems to be affecting the color too

blendSrc/blendDst does not matter when using blendEquation of min and max (webgl_materials_blending_custom )

i see, there was a chart somewhere in the threejs docs explaining all the various combinations. im sure one must work with color retention, if you find it i would love to use that in drei as well. btw i would recommend using meshLambert for the catcher plane/material, because it doesnā€™t get specular and envmap, which would also discolorize the result.

1 Like

Another update

  • using blendModes to make shadowMesh transparent did not work as expected :smiling_face_with_tear:
  • SoftShadowMaterial from AccumulativeShadows is working nicely ! :partying_face: :v:

tried the AccumulativeShadows style of rendering the shadows in the same scene again, but screen goes black and it restores itself but no shadows , so sticking to the brute force method of swapping meshes between the two scenes for now.

New problem :
When i clear the renderTarget using

    renderer.setClearColor("black", 1)
    renderer.clear()

it becomes fully opaque + black

but when i do

    renderer.setClearColor("white", 1)
    renderer.clear()

the plane is invisible as desired but the next time it accumulates shadows it takes a lot of frames(3x as usual) before the shadow mesh is visible again

any idea why ? @drcmda

not sure but i struggled with this a lot. i think starting from a opaque plane is OK, thatā€™s why i accumulate alpha. so basically divide 1 by the number of frames, start with 0 and add the dividends until you reach opacity 1 but just it is properly alphaTested.

1 Like

i think when i make the render target transparent the shadows/light do not get cast on it at full strengthā€¦

so we have to compute 4-5 times just to get the same result, when itā€™s opaque works nicely

when you say accumulate is this what you mean ?

// as frame index goes from 1 to params.frames, alphaTest goes from 0 to 1

softShadowMaterial.alphaTest=Math.max(0, MathUtils.mapLinear(index, 1, params.frames - 1, 0, 1))

shadow demo

  • Instead of computing the shadows in the render loop i compute them in a for loop directly with sleep function to reduce stress , is that the best way or itā€™s better to do compute one shadow frame per in each req animation frame ?

  • also any way to reduce the first frame freeze ? i did renderer.compile on the hidden scene but that freeze is still there

1 Like
function meshes_frustum_visible(item,mode){


if(mode==1){
item.traverse(function(child){
if(child.isMesh){
child.last_visible=child.visible;
child.visible=true;
child.last_frustumCulled=child.frustumCulled;
child.frustumCulled=false;
}
});
}
else{
item.traverse(function(child){
if(child.isMesh){
child.visible=child.last_visible;
child.frustumCulled=child.last_frustumCulled;
delete child.visible;
delete child.last_frustumCulled;
}
});
}


}


meshes_frustum_visible(scene,1);
renderer.render(scene,camera);
meshes_frustum_visible(scene,2);