Directional Light mapping in WebGPU

[Usnul]

I’ve struggled with getting a good-looking global illumination for Shade for a while. Methods that look good are slow, and methods that are fast don’t look good. And as you go further into getting something good looking that’s fast - the complexity goes through the roof.

I figured I’d give light maps a go, something I read a lot of literature on, and I already have a full-features ray tracer running on the GPU.

I didn’t want to just bake light maps, as that’s something you can do in the likes of Blender, and the result looks somewhat flat.

I wanted to have high dynamic range, which standard JPEG and PNG just don’t give you, and I wanted to capture directional component to irradiance, that is - when you look from a different angle - you get a different result, because the light you see if actually a reflection, and reflections have a directional component.

I thought lightmaps would be easy… Well, they are not. But before that, here are current results, this is just incoming indirect radiance at 128 samples per pixel:

here’s a ray traced reference of the scene

indirect means we don’t have direct light here, which is obvious when looking at the lit areas in the reference. On the other hand, you can see that areas lit indirectly have a lot of similarity. The reason we only capture indirect light is to be able to use light maps together with direct lighting.

if we focus on the gap between the two cubes, there’s a lot of green light reflected there, because the wall of the box is green, but if we look from the front into the same gap

we don’t see much of that same green, because most of the light is reflected from the back wall and the ceiling instead.

Not very impressive-looking so far. Very noisy and the UVs are far from perfect.

The Problem

There are essentially 3 challenges that need to be solved to have a light mapper

1. Generate UVs
2. Pack UVs for each mesh into a separate texture atlas
3. Bake

It might seem simple, but it really really isn’t

Generate UVs

So, UVs, how hard can it be? Considering there is a whole industry dedicated to just working with UVs - clearly, quite hard. Luckily for us, there’s xatlas, I read about the original thekla_atlas years and years back, always wanted to give it a go. Well, turns out there are problems even with that.

First, it’s C++, so you have to use WASM. Okay, smart people of Mozilla created xatlas-web. Slight issue, it only supports geometries that have less than 20k triangles, if you have more - you’re SoL, because someone hardcoded indices as Uint16. Why?..

web/xatlas-web.cpp

No problem, I know C++, I used emscripten a few times, we can fix this… turns out, recent version of emcc doesn’t work on Windows (where I happen to be). No problem, I can spin up a linux box. Turns out, recent version of emcc doesn’t compile this project. At this point we’re some 5 hours in, and I feel comitted.

After a bit of research, we find that xatlas-web was successfully compiled with emcc 2.0.7, huray, so we use a convenient docker image for that, and it works.

Now we have a working version.

Incidentally, why not use xatlas.js? because it’s built in a most convoluted way imaginable it seems, and the author still did the same mistake with Uint16.

Now everything is good right? - Wrong, whoever wrote xatlas-web clearly didn’t know what memory management is, because the library doesn’t clean up what it allocates, so you… hmm… slowly, run out of memory.

To save some pain, here’s working version by yours truly, and if you just want the binaries, they can be found here:

I’m so ready, let’s try it! … and it takes 140 seconds to generate UVs for Sponza… Not what I would call “suitable for real-time”.

After tweaking compile options we get this down to ~40s, which is still abysmal but… what can you do?

Now, xatlas does a teeny tiny little thing, it changes your topology… sometimes. That is, it will just decide to add a triangle here, or change the order of vertices there. What does this mean? It means the output you get is not usable directly, you have to either rebuild your geometry attributes and the index to fit the new topology, or backproject the UV for new topology onto the old topology.

I’ve had some success with backprojection, but then realized that it does actually make sense for xatlas to change your topology, because if you want unique and continuous UV space - some topologies just don’t allow that. I won’t go into detail as to why, but if you think about it a bit it should become obvious to you much quicker than it did to me, because you’re a smarter person, I assume.

So here we area, backprojection is a no-go as a general solution. The changes topology is broken on the other hand:

so, time for some magic

I wrote a dead-simple rasterizer, which will draw existing UV chart onto a fresh bitmap image, and track overlaps, if overlaps are few - we can say that UVs are non-overlapping and are suitable for light mapping. This works surprisingly well, and, from brief glances into xatlas code it seems this same technique is used in there in some places.

Problem with re-using existing UVs is that often UVs don’t make good use of available texture space, and we want them to make as close to “every pixel” as possible, because that rectangular region of UV will be given space in our final atlas, and any used pixels will just be wasted.

So, with a bit of extra magic, we re-scale the UVs to 0…1 range, nothing very impressive, but it works well.

Pack UVs

So we have UVs for a geometry, but that’s not quite enough. Our scene doesn’t have geometries, it has Meshes, and meshes can use the same geometry. This means we need an atlas that packs a light map per-mesh.
A simple way would be to just pick arbitrary grid, say 16x16 and pack every mesh light map into a square on the grid, we could pack
256 meshes that way, if we run out of space, we can just increase the grid.
The problem with that is that meshes can be of different size, imagine a mouse and an elephant. Mouse could use a much lower resolution light map than the elephant. What we want is a relatively uniform texel density for scene meshes in the light map.
So, naive grid-based packing is a no-go. I mean, you could do it, but even Sponza has 103 meshes

Most real-world scenes would just not work, you’d be wasting a UV space on tiny objects and your large objects would be severely low-res in the UV space.

Luckily for me, I already have an atlas packer written in meep, and it’s a good one, so all I had to do was write some hand-wavy world-size estimation heuristic for each mesh.

For comparisson, here’s a world-scaled packing, the colors represent texel density, in turbo scale

don’t mind red, it’s background color.

If I remove world scaling, forcing every UV set to be the same size, we get this:

Suddenly the walls get much lower texel density and a random book on the table gets huge amount texture space.

Here’s what that looks like baked

and again, here’s the world-scaled version

certainly not perfect, but much better.

Bake

Baking is the main thing of all of this, you can pack as well as you like, you can have perfect UVs, but if your bake is bad - you’re going to have a bad end-result.

What you want from a good bake are basically 3 things:

1. You want UV dilation, that is - you want to fill the areas around your UV islands with the same “color”, so that when texture is sampled on the edge of the UV - you get expected results, this is also important for mipmaps to work properly.
2. You want to accumulate a lot of samples
3. You want to denoise your output

I could add that your ray tracer needs to be good, but that’s a given.

So far, I’ve got none of these and my life is more difficult because the light maps are directional. Somehow it starts to make sense why lightmapping in a lot of engines is pretty bad and why directional lightmaps are not as common.

For those who are interested, the representation I’m going with is second order spherical harmonics (4 components) and I’m doing something similar to what metro exodus guys did, packing SH as:

L00 = YCoCg
L1_1 = Y
L10 = Y
L11 = Y

Basically, we encode full sh2 for the luminance, but actual chromacity (color) is encoded only for the l0 band, which is constant. It means you get a strong directional component for light intensity, but you don’t have any chroma variation. The trade-off gives you very compact representation though.

I’m using float16 x 6 for storage, but I suspect I could represent CoCg with u8 x 2 instead of float16 x 2 to save extra 2 bytes per texel.

I also apply deringing to the final SH, which takes care of negative lobes.

Why?

An important question is “why”, why do all of this? The answer is pretty straightforward - directional light maps approximate global illumination rather well, and they are very cheap at runtime, almost free in fact. You get both specular and diffuse components in there, and depending on your light map resolution, you even have pretty good specular reflections. The idea is to use this for product visualisation, arch-vis also comes to mind.

References

I believe activision guys wrote about directional light maps a fair bit in late 2010s, in case anyone wants to chase this up on their own. Might add references later on, for now this is mostly freeform and based on my own recollection.

[kpachinger]

@Usnul

it changes your topology

This works as a normal map, to parametrize effects. Ephemera (such as plasmatic flame) may be reduced to a sprite, but project volume. So the pipeline is multifaceted. Does lava reflect lava?

~ Talky “Tokamak Toucher” Tickertape Tragedy

[Usnul]

Worked some more, found that path tracer introduces fireflies (overblown pixels) when sampling the environment map, did the very shameful thing of clamping environment samples to some arbitrary value of luminance (10).

Noise has come down quite dramatically

And here’s the before

Found fireflies by tinkering the the shader, tried flagging up pixels with luma higher than X

Turned out that X could go as high as 100, meaning that these overblown pixels were about 100x brighter than the rest of the scene. Still not sure why they happen, but the clamp seems to be a common-enough solution.

Also added a few UV tweaks and started moving towards other parts, such as a proper denoiser, dilation of UVs and progressive baking.

Also tried a flat bake, that is - just a single irradiance value per pixel, and i must way it looks quite appealing too:

[Usnul]

Added conversative rasterization to avoid pixels on the edge of the UV layout sampling missing texel values

Before:

After:

it might be a bit subtle, but the edges are a lot cleaner now

Next, added a basic denoiser, repurposing a-torus one I already had for shadows and AO

This is far from perfect, and there’s no edge stopping function at all. I wanted to first see if denoising in Spherical Harmonic space would make sense at all. I thought a fair bit about, and I think it will fail in higher dimensions, but at sh2 it actually makes a lot of sense, as each coefficient of the SH corresponds to a cardinal direction (more or less).

I’d say this is a success, need to implement a sensible edge-stopping function, but it already looks pretty good, even if a bit over-blurred.

Next, not sure exactly why, but there’s pretty severe banding up-close on the sampled spherical harmonics, probably because of rounding in the coefficients or sharp signal boundaries during interpolation. A bit of gradient noise fixes it up quite well though

Before:

After:

And overall result so far

For comparison, here’s what I had yesterday

and here’s one more