Depth test not working when loading mesh dynamically

Questions
, ,
1

Using R3F, I show the player as a shadow when he is behind an object. This is achieved by rendering the object two times, the second time with depthTest=false and renderOrder=5:

export const useEntityShadow = (nodes: Mesh[], material: MeshStandardMaterial, asShadow?: boolean) => {
  const shadowMaterial = useMemo(() => {
    if (!asShadow) return material

    const shadowMaterial = material.clone()
    shadowMaterial.color = new Color(0, 0, 0)
    shadowMaterial.depthTest = false

    return shadowMaterial
  }, [asShadow, material])

  useEffect(() => {
    if (asShadow) {
      for (const node of nodes) {
        node.material = shadowMaterial
      }
    } else {
      for (const node of nodes) {
        node.renderOrder = 5
      }
    }
  }, [asShadow, material, nodes, shadowMaterial])

  return { renderOrder: asShadow ? 0 : 5, material: shadowMaterial }
}

This works fine, but if I unload/reload the environment objects, this effect stops working. Please check the attached video.

I have no idea what causes this, so any help would be appreciated. Thanks.