When using VSMShadowMap all shadow receivers will also cast shadows.
So castShadow not being true doesn’t matter then. I wonder if this has anything to do with it. Why does PCFSoftShadowMap look drastically better?
EDIT: Ah! That banding was shadow acne. I didn’t realize enabling VSM shadows forced shadow receivers to also cast shadow even if castShadow is false, so I had assumed the wall was not casting shadows on itself, leading me to believe it was some other issue!
To fix the banding, I had to adjust the light.shadow.camera.near/far values to tighten up the shadow camera frustum, then I could remove tiny amounts of acne with small shadow.bias values. The frustum near/far distance was previously way too large, which happened to work fine with PCFSoft shadows.
If I increase shadow.bias to a fairly larger number like -0.1, I get less shadow-map-edge-lines, but the whole shadow is almost gone, so that isn’t working:
A solution for the shadow outline artifact is to patch ShadowNode.prototype.setupShadowFilter to fade out shadows around the edges of the shadow map area, like so:
const originalSetupShadowFilter = THREE.ShadowNode.prototype.setupShadowFilter;
THREE.ShadowNode.prototype.setupShadowFilter = function ( builder, inputs ) {
// portion of shadow map UV space along edges to fade
const fadePortion = 0.05;
const base = originalSetupShadowFilter.call( this, builder, inputs );
const uv = inputs.shadowCoord.xy;
// Distance to nearest edge of the shadow UV box
const edgeDist = min( min( uv.x, uv.x.oneMinus() ), min( uv.y, uv.y.oneMinus() ) );
// Fade: 1 at the very edge, 0 inside (only if in a shadow)
const fade = sub( 1, smoothstep( 0, fadePortion, edgeDist ) );
return mix( base, 1, fade );
};