Three-gpu-pathtracer: A modular shader-based path tracing extension for three.js!

[gkjohnson]

Hello! Over the last several months I’ve been working on a high fidelity, physically based path tracer based on my three-mesh-bvh project and have just released the initial version! It includes material features such as transmission, textures, normal maps, metalness, GGX specularity, environment maps, tiled rendering, and more! There are a few demos to check out in the repo here:

GitHub - gkjohnson/three-gpu-pathtracer: Path tracing renderer and utilities...

It works by packing the BVH and geometry details into textures and implementing a raycasting algorithm in a shader to path trace the scene. There’s still a lot more to do to improve performance, compatibility with three.js meshes and materials, and generally improve the quality of the project so I welcome anyone who is interested in contributing to come learn with me! I’m happy to offer guidance and point people in the right direction to get something implemented. And of course any feedback is welcome! I’m looking forward to seeing the project grow.

And as always please do share if you wind up building something with it!

transmissive gelatinous cube

material demos

Lego models, of course

other demo models

Garrett

[Usnul]

Very pretty, well done!

[Yokiano]

It looks awesome.
In your opinion what would be the main use cases for something like this on the web?

[gkjohnson]

All kinds of things, I think. Architecture visualization, product visualization / configuration viewers, real time light map or ao map baking, scene editor rendering, procedural three.js art rendering, among others!

[gkjohnson]

Here are some new demos and test models! I’ve added a gem / jewelry demo model which you can check out here:

Jewelry Demo

And an interior scene with a lot of reflective, refractive, and metal surfaces lit entirely from emissive surfaces:

Interior Demo

[gkjohnson]

The second version of three-gpu-pathtracer has just been released!

Release v0.0.2 · gkjohnson/three-gpu-pathtracer

It includes a few new features including depth of field support and support for skinned and morph target meshes.

New Demos

Depth of Field Demo

Skinned Geometry Demo

Morph Target Demo

Depth of Field

Animated Geometry

[fraguada]

I’ve been watching your posts on Twitter and ran some of the demos… Very impressive! My next thought goes to denoisers. I’ve tried a few that are available in js but haven’t been impressed since I’m a bit spoiled playing with Intel, Nvidia, and AMD denoisers. I was curious if Intel’s open image denoiser could be compiled to wasm, but the response I got was that no one was working on it.

Do you have any thoughts on denoisers in js to compliment your pathtracer?

[gkjohnson]

The next release will include env map importance sampling which should help things resolve more quickly even without a denoiser. Even so a denoiser would be nice. The “SVGF” denoiser is the one I’ve been looking in to. I have an issue here to track it if you’d like to help:

Add denoiser

In terms of WASM - I don’t think that’s the way to go. I think we can learn from open source implementations but my impression is that these types of denoising will have to be done with shaders so I’m not sure how much WASM will really afford here.

[gkjohnson]

I’ve just released v0.0.3 of three-gpu-pathtracer! The most significant changes include environment map importance sampling which prioritizes sampling brighter pixels in the environment to enable faster & smoother image convergence with sharp environment maps, support for transparent background rendering, and more! There are some breaking changes in this release so take note.

Release v0.0.3 · gkjohnson/three-gpu-pathtracer

renders with harsh shadows enabled by environment importance sampling

transparent backgrounds

[gkjohnson]

I just released v0.0.4 of three-gpu-pathtracer! This release includes a number of fixes, improve texture offset support for materials, orthographic rendering, and more!

Full release notes here:

Release v0.0.4 · gkjohnson/three-gpu-pathtracer

orthographic rendering

[gkjohnson]

v0.0.5 has now been released! It comes with some big new features like clearcoat material properties, area light support, and equirect environment map rendering!

Area light demo here!

Release v0.0.5 · gkjohnson/three-gpu-pathtracer

[Usnul]

Looks nice, but I think you might have some problems in the convolution step:

there’s 1k samples here, and you can see some very suspect pixels around the right leg. One of those (the left-most) popped white around 900 sample count, after having relatively common gradient with the neighbouring pixels.

[gkjohnson]

v0.0.6 has been released! This new version includes support for spot lights, circular area lights, and some new material properties!

Release Notes and demos here

material properties

circular area lights

spot lights

[mjurczyk]

Glancing a bit into the future — Average Temporal Sampling Fan + Average De-noising Enjoyer = Gigachad Almost-perfect Real-time Path-tracing in the browser ?

Or is the de-noising not-so-real-time?

[gkjohnson]

Oh there’s some good stuff coming. The contributions coming in are amazing.

Or is the de-noising not-so-real-time?

Desnoisers can be real time but there’s a lot of different approaches to denoising - some of which are more complicated than others. But that’ll come eventually I think. All progress!

[makc3d]

I tried to make sense of all those fancy denoiser params at Shader - Shadertoy BETA but could not

If I just rEmOvE tHeM it still works just fine:

vec4 smartDeNoise(sampler2D tex, vec2 uv, float radius, float threshold)
{
    float radQ = radius * radius;

    vec4 centrPx = texture(tex,uv);
    
    float zBuff = 0.0;
    vec4 aBuff = vec4(0.0);
    vec2 size = vec2(textureSize(tex, 0));
    
    for(float x=-radius; x <= radius; x++) {
        float pt = sqrt(radQ-x*x);  // pt = yRadius: have circular trend
        for(float y=-pt; y <= pt; y++) {
            vec2 d = vec2(x,y);

            vec4 walkPx =  texture(tex,uv+d/size);

            vec4 dC = walkPx-centrPx;
            float deltaFactor = exp( -dot(dC, dC) * threshold);
                                 
            zBuff += deltaFactor;
            aBuff += deltaFactor*walkPx;
        }
    }
    return aBuff/zBuff;
}

void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
    // Normalized pixel coordinates (from 0 to 1)
    vec2 uv = fragCoord/iResolution.xy;

    float szSlide = .001;        
    vec2 mouse =iMouse.xy/ iResolution.xy;
    if(mouse.x <= 0.0 &&  mouse.y <= 0.0) mouse.x = .5; // to show initial screen half splitted
    
	fragColor = uv.x<mouse.x-szSlide  ? texture(iChannel0,uv)
          : (uv.x>mouse.x+szSlide ? smartDeNoise(iChannel0, uv, 10.0, 100.0) // <- change here
          :  vec4(1.0));
    
}

[gkjohnson]

The glsl smart denoiser is the same one that was implemented in this PR. It’s good for low frequency noise but there are better solutions (some mores listed here).

[gkjohnson]

v0.0.7 has been released! New features include a denoiser material, transmission attenuation support, vertex colors, thin film transparency, and improved material sampling behavior!

Release notes here:

Release v0.0.7 · gkjohnson/three-gpu-pathtracer

Volume Attenuated Transmission

Thin Film Transmission

[gkjohnson]

I’ve just released v0.0.8! There are quite a few memory-related fixes for the project, the ability to provide a separate background image from lighting, other quality of life improvements, and - most importantly - significant quality improvements to specular surface sampling! Specifically the fresnel term for fresnel has been adjust to fix an issue where mid-range rough and metallic materials were being rendered as overly-bright.

Full release notes here:

Release v0.0.8 · gkjohnson/three-gpu-pathtracer

Custom Background Image

Specular Fresnel Lighting Comparions (v0.0.7 vs v0.0.8)

[Harold]

Not sure if it’s just me, but the latest 2 examples shown on the github page complete crash my graphics driver on an RTX 2080 Ti, latest drivers / Intel i7 13700k.

Other examples don’t work anymore either. Normal WebGL things run just fine.

I’m using Brave browser, which by extent is just chromium.