Dr.Strangelight or: How I Learned to Stop Worrying and Love the Cluster

[Fyrestar]

At first about 40-50 FPS at QHD windowed with a TitanX

What is interessting, i had a file downloaded with the downloads bar at bottom in chrome that makes the viewport smaller, after closing it, also after reloading i expected it to drop a little further, instead it suddenly was mostly stable around 50-56 FPS also at fullscreen.

But it does make the GPU workload go fairly high unfortunately, with deferred lights and the same amount of lights instanced it barely makes any difference than without lights.

[Usnul]

@looeee

The actual demo is very cool though, great work! It’s running at 4k@120fps with 60% gpu use on a 3070m

That’s really decent

@looeee

Would love to see this added to three.js someday.

I highly doubt that, too much complexity Although I would want to t see a sophisticated turn-key rendering solution, I’ve accepted that three.js isn’t headed in that directly. Which isn’t necessarily a bad thing either, over the years it’s become easier to extend three.js and build on top of it as some sort of mid-level API.

@Fyrestar

At first about 40-50 FPS at QHD windowed with a TitanX

That’s 1000 generation, or the 900? I’m not sure what the frame of reference is. Never had the pleasure of working with one of those beasties

@Fyrestar

But it does make the GPU workload go fairly high unfortunately, with deferred lights and the same amount of lights instanced it barely makes any difference than without lights.

You mean doing lighting on a G-buffer? Yeah, that would be faster.

Overall I ran into a bit of a wall here, my implementation is… “elastic”, it can be configured to any cluster resolution, such as 32x32x32 clusters or even 1x1x1 which would be equivalent of not having clustered benefits at all. The issue is light assignment to clusters. More clusters you have - less work your GPU needs to do, as there will be fewer lights binned to a cluster. At the same time, assigning lights to more clusters takes more CPU work.

I’ve optimized it to a fairly extreme degree, there’s a custom BVH implementation written specifically for it, most of the code is inlined, almost all of the data is in typed buffers, even forcing shared ArrayBuffers for ensure better data locality. Heck, I even did register pressure optimizations in JS code. I’m fairly positive that there’s little room left for optimization there. Yet still, each cluster assignment takes non-trivial amount of time.

At this point, I’ve considered moving the whole “visible lights” index as a BVH directly to GPU, bypassing clustering altogether, but that creates its own problems because of extra loops and branches.

I look forward to GPU compute coming in WebGPU next year, doing cluster light assignment on GPU would remove this bottleneck.

For now I’m pretty happy, the example with Sponza is actually a pretty bad case for my engine, as it’s currently tuned more towards 100-ish lights per view and with little overlap between lights. Sponza demo uses fairly large lights and they are quite densely packed, resulting in large number of light per cluster.

Thanks for the comments

[Usnul]

Btw, you can try changing cluster resolution to see how it affects the GPU-side of things via console like so:

lights.setTileMapResolution(x,y,z);

I recommend 20,9,20 (3600 clusters), it’s higher than the default 24,14,8 (2688 cluster) and exploits somewhat large depth of the scene better.

[Fyrestar]

You did a great job on this, there isn’t rally some other solution to forward lighting currently especially for bandwidth limited devices that could be very interesting. However, anything just as this is better than the default way of all shaders getting all existing lights assigned and computing them all even if not affected, which is rather suited for single spot scenes or a directional sun.

@Usnul

I look forward to GPU compute coming in WebGPU next year, doing cluster light assignment on GPU would remove this bottleneck.

I don’t know how the technique works in detail, but could the cluster get constructed in one or more render targets?

@Usnul

You mean doing lighting on a G-buffer? Yeah, that would be faster.

Yes with a g-buffer, i don’t see an alternative for it (also considering the scale of my world) but it doesn’t hurt on modern and average good enough cards, but the bandwidth consumption is higher, i even need a position attachment (as UE and others use too afaik) as for the distance/scale of the world a accurate non-flickering reconstruction of the coordinates from depth and camera matrix is not possible.

In my game you can also build houses like in The Sims, i considered making a isolated house/property-only mode possible for weaker mobile devices without the full massive open world, i think this technique could be a perfect match for that scenario for typical room lights, some other light sources here and there but not overwhelming and mostly static and a quite predictable volume of the scene, it sure would remove the bandwidth concerns with mobiles that come with deferred lights.

[Usnul]

@Fyrestar

I don’t know how the technique works in detail, but could the cluster get constructed in one or more render targets?

It’s possible, but it’s a hassle. You can’t render to a 3d texture (which is what cluster texture is), and that means you have you unroll it first, and then pipe it back to CPU from GPU because you can’t re-cast the same buffer, at least not via three.js. All that would probably still be much faster for bad scenes, and probably will have tiny overhead in general, but it’s just the ugliness of doing all that that puts me off, oh and having to write BVH walking code for GLSL. Maybe one day in the future, if the usage patterns require it

@Fyrestar

Yes with a g-buffer, i don’t see an alternative for it (also considering the scale of my world) but it doesn’t hurt on modern and average good enough cards, but the bandwidth consumption is higher, i even need a position attachment (as UE and others use too afaik) as for the distance/scale of the world a accurate non-flickering reconstruction of the coordinates from depth and camera matrix is not possible.

Yep, UE uses a lot of buffers, they don’t seem to have an issue with that. It’s quite the marvel really. I found that buffers don’t have a huge overhead too, it’s more the sheer number of pixels you end up shading, and not so much the bandwidth in my experience. Then again, perhaps my usecases are non-standard.

@Fyrestar

In my game you can also build houses like in The Sims, i considered making a isolated house/property-only mode possible for weaker mobile devices without the full massive open world, i think this technique could be a perfect match for that scenario for typical room lights, some other light sources here and there but not overwhelming and mostly static and a quite predictable volume of the scene, it sure would remove the bandwidth concerns with mobiles that come with deferred lights.

That sounds really cool, I look forward to seeing your game.

I should qualify that my solution does light culling before cluster assignment, so having thousands of lights in the scene is not really an issue. It’s how many are in the frustum - that’s the bottleneck. Then again, we both use spatial indices as a basis of your engines, so no surprises there

[Fyrestar]

@Usnul

It’s possible, but it’s a hassle. You can’t render to a 3d texture (which is what cluster texture is), and that means you have you unroll it first, and then pipe it back to CPU from GPU because you can’t re-cast the same buffer, at least not via three.js. All that would probably still be much faster for bad scenes, and probably will have tiny overhead in general, but it’s just the ugliness of doing all that that puts me off, oh and having to write BVH walking code for GLSL. Maybe one day in the future, if the usage patterns require it

Ah yes, i believe WebGL2 does support it though but it’s not implemented in THREE anyway. Does that cluster need interpolation? Otherwise it might could be an atlas with sliced tiles.

@Usnul

Yep, UE uses a lot of buffers, they don’t seem to have an issue with that. It’s quite the marvel really

I saw a video regarding some other tech where the buffers used were also shown, it’s not some forced standard, but for specific features like deferred lights etc they kick in. There was depth, position, normals, edge normals or such (for beveling edges), material id’s and/or object id’s and some further attributes for PBR. Quite a huge set that feels like transfering a entire movie every frame, i think we are simply over that hill of it being a real issue with how fast the cards power improved and how fast all consumers got hands on new hardware.

@Usnul

That sounds really cool, I look forward to seeing you game.