Tone Mapping Overview

[Usnul]

While working on Shade, I’ve been doing research on tone mapping and I wanted to share some findings.

First, let’s get a good starting point, we have a cartoony scene with a basic directional light normalized to 1.0 intensity

We are not doing any kind of tone mapping here, and it looks too dark. Lets add some intensity to the light to make it look good

Here’s intensity = 2

Still too dark, lets crank it up to 10

This looks good, but we have a problem now, some areas are over-saturated, that is - color is so bright it just ends up as being white, through the magic of Photoshop, here’s an area of solid white in the image (highlighted in red):

Now, this is a purely diffuse image, that is - roughness is set to 1 for everything here, it’s a cartoony scene. Let’s try the same with a scene that has some glossy materials

Again, we’ll start with light intensity set to 1

Let’s bring up light intensity to something more visually interesting, say 10 like last time:

Okay, we have a lot more brightness on the wood, and the rubber looks more readable, but we can easily see that large parts of the scene are blown out, here’s the Photoshop magic again to highlight areas with FFFFFF (solid white)

And here’s the same thing but luminance mapped to a false color

You can clearly see areas that are close to or at clipping range.

Right, so this is the main issue, our screens can generally only reproduce light intensity between 0 and 255 for each color channel, but realistic lighting and materials just don’t care about such trivial things.

Why the Tone Mapping?

Our renderers are producing what’s referred to as HDR images, High Dynamic Range, basically we mean that values can go “High”, or more speicically higher than the fixed 0-255 range that our hardware can do.

Solution to this is compression, we take a larger range of intensities and compress them to what the display can reproduce. In mathematical terms, we don’t actually care about the 8bit stuff, we just treat 0-255 range as 0 - 1 range. The compression refers to taking values much higher than 1 and ensuring they fit into this 0-1 range, hence “compressing”, we take a “big” range and “compress” it to smaller range.

To illustrate this a bit better, he’s a curve for ACES tone mapping (compression):

You’ll notice that values for 0.1 to 1.0 “brightness” are taking up majority of hte output range, that is - most of the original colors are mapped very similarly, but colors that would be clipped to just solid white or be too dark to tell apart and clamp to black or near-black are given a lot more range on the output scale.

This makes it so that very dark areas are still visible and you can tell them apart somewhat, and very bright areas are given some space on the output curve as well, up to ~100x brighter than what you’d get without mapping

Isn’t ACES the answer?

Now, with motivation out of the way, here are, we need tonemapping. And most of the industry has agreed on ACES tonemapping for years now, here’s what that looks like for our earlier scene:

For comparisson, here’s the one without any color mapping again:

There is a problem with ACES though, it has a very distinct look to it, and it suffers from degradation towards primary 6 colors, here’s an illustration with range:

As you can see, it devolves into banding each color into one of distinct 6 colors without much mixing. So your oranges will end up yellow and your water will have a cyan color. Let’s crank light intensity up to 100 on the previous scene to have a look at these artifacts

As you can see, oranges have disappeared, and hues look very much clamped. There is no variation at all. We do still have gradients in intensity, but not in hue.

To help illustate, here’s the hue of the ACES mapped scene at 10 light intensity

and here it is at 100 light intensity

Here’s are the unmapped hues

If you take nothing else away from this about ACES, take away the fact that ACES changes your colors, not just the intensities.

To make it a bit more clear, here’s what pure hue sweep look like without tone mapping from intensity 0 to 3

as expected, color blows out and clips without mapping

and here it is with ACES

As you can see the color bands become noticeable. Yellows dominate more and more, as well as cyans and pinks.

Son, I am a confuse

So, using ACES actually modifies how colors in your output image looks. Meaning that ACES creates a certain color feel to your images.

Later on, we got AGX, which is intended to be more neutral and preserve colors better.

Recently a tonemapper was proposed for more accurate color reproduction, which is now termed “Khronos PBR Neutral”

Now, if there are so many tone mappers out there, what is a humble graphics engineer to do? Which is the “best”? What is the right answer?

The answer is - it depends. I came to terms with the fact that color is subjective, and tone mapping is as much art as it is science. Sometimes you want to create a specific look or a mood, and tone mapping can help or hurt there.

“The Answer”

To help, here are 4 color mappers that you are likely to encounter or want to use. I’m throwing “no mapping” into the mix as well for a complete picture.

• None - no tone mapping, reference
• Reinhard - luma-based Reinhard operator
• ACES
• AGX
• Khronos PBR Neutral

Hue reproduction

Color sweep is for values between 0 and 10 in intensity. Formula is as follows:

hsv_to_rgb( vec3( uv.y, 0.8, 1.0 ) ) * ( uv.x * 10.0)

None

Reinhard

ACES

AGX

Khronos PBR Neutral

Rainbow

Formula

hsv_to_rgb(vec3(uv.y, 1.0 - uv.x, 1.0))

None

Reinhard

ACES

AGX

Khronos PBR Neutral

Dark sweep

Color values in intensity range of 0 - 0.1, formula:

hsv_to_rgb(vec3(uv.y, 1.0, mix(0.0, 0.1, uv.x)))

None

Reinhard

ACES

AGX

Khronos PBR Neutral

Pica Pica scene

at 2.2 light intensity and light temperature of 5500 K

None

Reinhard

ACES

AGX

Khronos PBR Neutral

Sea Keep scene

at 10 light intensity and light temperature of 5500 K

None

Reinhard

ACES

AGX

Khronos PBR Neutral

Flight Helmet scene

at 2.2 light intensity and light temperature of 5500 K

None

Reinhard

ACES

AGX

Khronos PBR Neutral

Bistro scene

at 5.2 light intensity and light temperature of 5500 K

None

Reinhard

ACES

AGX

Khronos PBR Neutral

Opinions

To sum it up. If you have any kind of realistic lighting in your scene - I recommend not going with the “None” option, you need some kind of tone mapping. If you know that light will never clip - that’s still an option though.

When in doubt - ACES is still good, but it will create a certain look for you. In general it boosts contrast and tends to darken your image.

If you are making a visualiser for a product where accurate color reproduction is important, or you are planning to embed your 3d into a 2d page - Khronos PBR Neutral is a solid choice, as it will faithfully reproduce colors. You can fairly confidently use customer’s color palette and expect it to look the same after rendering.

If you just want a tone mapper, but you don’t want to add a specific artistic look to your final image - AGX is a good choice. Although, because it is not dramatic like ACES - your scene will look a bit flat by comparrison. This not a bug but a feature though.

Disclamar

• All screenshots were taken in Shade
• ACES implementation is extremely close to three.js implementation, color matrices are exactly the same
• AGX implementation is close to three.js, but it’s following Blender more closely
• All scene screenshots have SSR, SSAO and Bloom on
• Gamma correction is outside of the scope of this article
• Color space is rec 709

References

In no particular order

• Tone mapping - Wikipedia
• PBR Neutral Tone Mapping
• Minimal AgX Implementation | IOLITE
• Add AgX tonemapper by bejado · Pull Request #7236 · google/filament · GitHub
• Press Release - The Khronos Group Inc

[Fennec]

At first, I thought this would be one of those complex graphic engineering topics where half of it would go over my head, but it turned out to be very informative and easy to follow, thanks for that. You have a knack for breaking down complex concepts and making them understandable, even in the areas I still don’t fully grasp.

As you pointed out, it’s more a matter of preference, mood, or taste. Based on the examples you’ve shared, I feel that Khronos PBR mapping has the best feel. Are there any plans to integrate it with three.js?

[Usnul]

Perhaps in the future, on my end it’s all just shader code with a bit of convenience to glue the code together. I might be weird in this, but the more I learn - the less I see framework or library as something sacred, it’s there, but it’s just an abstraction.

That, plus I plan to offer some kind of integration with three.js in the future.
I think three.js does an amazing job for the user from perspective of offering a very simple, yet powerful, API.

[user123]

Excellent post, very informative, thank you.

Are there any performance implications between the different options?

[Usnul]

@user123

Are there any performance implications between the different options?

Not really, tone mapping is so cheap these days that it’s practically free. There are only a couple of matrix operations at most, and 3x3 ones at that, not the 4x4 that we’re used to. There are no texture fetches either.

There are different ways of doing the same thing, some are working on curves, some are working on matrices, some are using lookup tables (very small ones, so still cheap), but it’s more of an implementation detail rather than anything related to a specific tone mapping operation itself, such as ACES or AGX.

For example, here’s the core of ACES:

const  ACESInputMat = mat3x3(
		vec3( 0.59719, 0.07600, 0.02840 ),
		vec3( 0.35458, 0.90834, 0.13383 ),
		vec3( 0.04823, 0.01566, 0.83777 )
);
const  ACESOutputMat = mat3x3(
		vec3(  1.60475, -0.10208, -0.00327 ),
		vec3( -0.53108,  1.10813, -0.07276 ),
		vec3( -0.07367, -0.00605,  1.07602 )
);


color = ACESInputMat * color;
color = ACESOutputMat * color;     

There’s more to it, but that would be common with any tone mapper, so it’s basically just 2 mat3x3 * vec3 operations.

If I was to say something about performance cost, I’d say that older techniques tend to be faster, just because we, as engineers, spent more effort thinking of ways to make them faster, with that, ACES and Reinhard are going to be fastest generally. But again, we’re talking about very low overhead to begin with.

[elliotjames]

This forum is amazing!

It’s excellent and very easy to understand.
I’m new here, but I can already tell there’s so much I can learn about Three.js.

Looking forward to your next post!

[GitHubDragonFly]

I think that Khronos PBR is actually THREE.NeutralToneMapping.

You could possibly try one of my viewers, maybe GLTF Viewer whose picture is attached.

Just make sure to click the Eq button once the model is loaded.

You could also use the BGND Browse button to manually load some other equirectangular texture from a computer (while still keeping the Eq button enabled). See the next picture with the san_giuseppe_bridge_4k.hdr which I found as available in one of the GitHub repositories while three.js has its 2k version.

[Fennec]

@GitHubDragonFly

I think that Khronos PBR is actually THREE.NeutralToneMapping.

Yep! It’s called Khronos PBR Neutral, so it probably is or at least the closest alternative. Thanks for the heads up!

[Usnul]

yep, here’s more

KhronosGroup/ToneMapping/blob/b5a2eed5ddf6c2227090449399de9c7affb9e4c9/PBR_Neutral/README.md#pbr-neutral-specification