Hi all. I’ve been looking at the examples within the Three.js webpage and I stumbled across this: example
I know this is using a library which enables path tracing for the scene it is applied to, but I would like to know if the lightning, and reflections are also part of the library script. I’ve been investigating the code and I’m sure the shadows are part of the library, but have not been able to figure out if the lightning and reflections are.
It’s not clear what you’re asking. You cannot have shadows without lighting. And reflections are a part of the material model implemented in the project. Lighting can be implemented through environment maps, point / directional lights, area lights, or emissive materials. Though emissive materials will take significantly longer to render. The demo you’ve linked uses exclusively emissive materials for lighting.
Hi @gkjohnson, thank you very much for your response. My apologies if my question was not clear, I’m very new to Three.js and 3D.
I would like to ask some questions to clarify my path. ‘Material model’ means the materials implemented by three-gpu-pathtracing library, right?
‘Emissive materials’ means materials with the emissive property being enabled? In that case, the material lighting is independent from the outside lighting, right?
I’ve been trying to use this library for a project that allows the user to dinamically position GLB on a scene, to later click a button and render the scene with path tracing. This project does not use react-three-fiber, it uses plain React and Three, and I’m losing my mind on how to implement it. Could I get some help? This is how my render function looks like.
// My function to setup path tracing..
setupPathTracing () {
this.ptMaterial = new PhysicalPathTracingMaterial();
this.ptRenderer = new PathTracingRenderer( this.renderer );
this.ptRenderer.setSize( window.innerWidth, window.innerHeight );
this.ptRenderer.camera = this.camera;
this.ptRenderer.material = this.ptMaterial;
this.fsQuad = new FullScreenQuad( new MeshBasicMaterial( {
map: this.ptRenderer.target.texture
} ) );
this.scene3D.updateMatrixWorld();
this.generator = new PathTracingSceneGenerator();
// this.generator = new DynamicPathTracingSceneGenerator();
const { bvh, textures, materials, lights } = this.generator.generate( this.scene3D );
const geometry = bvh.geometry;
this.ptMaterial.bvh.updateFrom( bvh );
// update bvh and geometry attribute textures
this.ptMaterial.bvh.updateFrom( bvh );
this.ptMaterial.attributesArray.updateFrom(
geometry.attributes.normal,
geometry.attributes.tangent,
geometry.attributes.uv,
geometry.attributes.color,
);
// update materials and texture arrays
this.ptMaterial.materialIndexAttribute.updateFrom( geometry.attributes.materialIndex );
this.ptMaterial.textures.setTextures( this.renderer, 2048, 2048, textures );
this.ptMaterial.materials.updateFrom( materials, textures );
// update the lights
this.ptMaterial.lights.updateFrom( lights );
};
// Use this instead of usual render if path tracing mode is enabled
renderWithPathTracing () {
this.ptRenderer.reset();
this.camera.updateMatrixWorld();
this.ptRenderer.update();
// if using alpha = true then the target texture will change every frame
// so we must retrieve it before render.
this.fsQuad.material.map = this.ptRenderer.target.texture;
// copy the current state of the path tracer to canvas to display
this.fsQuad.render( this.renderer );
};
...
// Render loop inside componentDidUpdate
let render = () => {
//** UPDATE CAMERAS */
orbitController.update();
this.camera.updateMatrix();
this.camera.updateMatrixWorld();
cubeCamera.update( this.renderer, this.scene3D );
for ( let elemID in planData.sceneGraph.LODs ) {
planData.sceneGraph.LODs[ elemID ].update( this.camera );
}
if ( this.props.isPathTracing ) {
if ( this.ptRenderer === null || this.ptMaterial === null ) {
this.setupPathTracing();
}
this.renderWithPathTracing();
} else {
this.renderer.render( this.scene3D, this.camera );
}
this.renderingID = requestAnimationFrame( render );
};
render();
When I execute this and I enable path tracing mode, the render view turns black and I get the following error on console:
three.module.js:19016 THREE.WebGLProgram: Program Info Log: C:\fakepath(3817,10-36): warning X3577: value cannot be NaN, isnan() may not be necessary. /Gis may force isnan() to be performed
C:\fakepath(3817,10-36): warning X3577: value cannot be NaN, isnan() may not be necessary. /Gis may force isnan() to be performed
C:\fakepath(1329,1-6): warning X4000: use of potentially uninitialized variable (dyn_index_vec3_int)
error X8000: D3D11 Internal Compiler Error: Invalid Bytecode: Index Dimension 1 out of range (4904 specified, max allowed is 31) for operand #2 of opcode #5078 (counts are 1-based). Aborting.
error X8000: D3D11 Internal Compiler Error: Invalid Bytecode: Can't continue validation - aborting.
Warning: D3D shader compilation failed with default flags. (ps_5_0)
Retrying with skip validation
C:\fakepath(3817,10-36): warning X3577: value cannot be NaN, isnan() may not be necessary. /Gis may force isnan() to be performed
C:\fakepath(3817,10-36): warning X3577: value cannot be NaN, isnan() may not be necessary. /Gis may force isnan() to be performed
C:\fakepath(1329,1-6): warning X4000: use of potentially uninitialized variable (dyn_index_vec3_int)
error X8000: D3D11 Internal Compiler Error: Invalid Bytecode: Index Dimension 1 out of range (4904 specified, max allowed is 31) for operand #2 of opcode #5078 (counts are 1-based). Aborting.
error X8000: D3D11 Internal Compiler Error: Invalid Bytecode: Can't continue validation - aborting.
Warning: D3D shader compilation failed with skip validation flags. (ps_5_0)
Retrying with skip optimization
C:\fakepath(3817,10-36): warning X3577: value cannot be NaN, isnan() may not be necessary. /Gis may force isnan() to be performed
C:\fakepath(3817,10-36): warning X3577: value cannot be NaN, isnan() may not be necessary. /Gis may force isnan() to be performed
C:\fakepath(3817,10-36): warning X3577: value cannot be NaN, isnan() may not be necessary. /Gis may force isnan() to be performed
C:\fakepath(3817,10-36): warning X3577: value cannot be NaN, isnan() may not be necessary. /Gis may force isnan() to be performed
C:\fakepath(1329,1-6): warning X4000: use of potentially uninitialized variable (dyn_index_vec3_int)
C:\fakepath(2160,1): warning X4000: use of potentially uninitialized variable (f_intersectsRectangle)
C:\fakepath(2161,1): warning X4000: use of potentially uninitialized variable (f_intersectsRectangle)
C:\fakepath(2162,1): warning X4000: use of potentially uninitialized variable (f_intersectsRectangle)
C:\fakepath(2178,1): warning X4000: use of potentially uninitialized variable (f_intersectsCircle)
C:\fakepath(2179,1): warning X4000: use of potentially uninitialized variable (f_intersectsCircle)
C:\fakepath(2825,1): warning X4000: use of potentially uninitialized variable (f_envMapDirectionPdf)
C:\fakepath(2272,1): warning X4000: use of potentially uninitialized variable (f_ggxDistribution)
C:\fakepath(2383,1): warning X4000: use of potentially uninitialized variable (f_evalIridescence)
C:\fakepath(2523,1): warning X4000: use of potentially uninitialized variable (f_transmissionEval__SurfaceRec)
C:\fakepath(2272,1): warning X4000: use of potentially uninitialized variable (f_ggxDistribution)
C:\fakepath(1329,1-6): warning X4000: use of potentially uninitialized variable (dyn_index_vec3_int)
C:\fakepath(2272,1): warning X4000: use of potentially uninitialized variable (f_ggxDistribution)
C:\fakepath(2383,1): warning X4000: use of potentially uninitialized variable (f_evalIridescence)
C:\fakepath(2523,1): warning X4000: use of potentially uninitialized variable (f_transmissionEval__SurfaceRec)
C:\fakepath(2272,1): warning X4000: use of potentially uninitialized variable (f_ggxDistribution)
C:\fakepath(2825,1): warning X4000: use of potentially uninitialized variable (f_envMapDirectionPdf)
C:\fakepath(1329,1-6): warning X4000: use of potentially uninitialized variable (dyn_index_vec3_int)
C:\fakepath(2272,1): warning X4000: use of potentially uninitialized variable (f_ggxDistribution)
C:\fakepath(2383,1): warning X4000: use of potentially uninitialized variable (f_evalIridescence)
C:\fakepath(2523,1): warning X4000: use of potentially uninitialized variable (f_transmissionEval__SurfaceRec)
C:\fakepath(2272,1): warning X4000: use of potentially uninitialized variable (f_ggxDistribution)
Any help would be appreciated, I’m looking forward to learning more on this matter, but right now I’m stuck trying to implement it.
I’ve just published a v0.0.12 version of the project with a workaround for a shader compilation issue in D3D11 on Windows which is what your errors look like. Give it another try.
This was the cause of my immense stack trace. Thanks a lot!
However, I’m still getting a black screen when I try to run the script that enables path tracing from your library.
Any ideas on what might be wrong with my code? I may be mixing up in which order does the vanilla Three and three-gpu-pathtracer code have to executed.
I seem to be getting this error sometimes still:
THREE.WebGLProgram: Program Info Log: C:\fakepath(3832,10-36): warning X3577: value cannot be NaN, isnan() may not be necessary. /Gis may force isnan() to be performed
C:\fakepath(3832,10-36): warning X3577: value cannot be NaN, isnan() may not be necessary. /Gis may force isnan() to be performed
C:\fakepath(1329,1-6): warning X4000: use of potentially uninitialized variable (dyn_index_vec3_int)
I don’t see where lights or an environment map has been added to the renderer which are required for the scene to show up as non-black. Otherwise please make a js fiddle or live repro case so it’s easier to see what’s happening.
Environment map and lights are added, but I do not think this is about lightning. I will share my whole component where all the Three renderer is handled along with my attempt to implement three-gpu-pathtracer. The code does not work since this component takes part of a large app and its importing stuff from a lot of modules plus props passed down the component three.
Anything I can do to clarify this and help you discern what is going on, please tell me. I’m handling all the logic from the library on setupPathTracing and renderWithPathTracing. Any small amount of info on what I’m trying to do will help me, specially on if what I’m trying to do is possible or not.
Edit: I will go ahead and clarify the context. What the relevant part of the app is trying to do is load a scene in 3D with several GLB loaded. The scene has OrbitControls and you can select those GLB to apply morph event to them. I’m really interested on giving the option to render that same scene with the path tracing functionality of your library. That is done with a button changing the variable called isPathTracing passed down as props to the component from the Fiddle.
Edit 2: I would also like to know if this library is restricted to using certain kind of Three.js lights, materials, geometries or shadows. This would also help me a lot on knowing if I can tell my team to continue further with the project while we figure out how to implement this with to our app logic. It would be a shame to work on the app with certain materials or textures to find out 6 months later that those were not compatible with your library.
I would also like to know if this library is restricted to using certain kind of Three.js lights, materials, geometries or shadows.
The project works with area lights, point lights, directional lights, spot lights, and environment lighting. Emissive surfaces are supported but with the caveats mentioned above. InstancedMeshes and InterleavedAttributeBuffers are not supported. And only MeshStandardMaterial and MeshPhysicalMaterial are supported - generally custom shaders are not intended to be supported.
In terms of a JSFiddle - I’d need something that’s running. Just looking at a copied set of a code out of context is not helpful since anything else could be wrong and I can’t test any code paths to understand what is going wrong. My recommendation is to make sure you can get the implementation working in a vanilla JS context before integrating with React or at least getting an example running with an error so people can help. Codesandbox may be better for running react code compared to jsfiddle.
Hi @gkjohnson. Thanks a lot for your response! You’ve been of great help already.
I will work on giving you a working copy of our implementation so that we can altogether trace what the issue is with our logic. I’ll get back to you as soon as I have it working.
I am very new to Three so I want to make sure I understand what we can’t do. I see that InstancedMeshes and InterleavedBufferAttributes are not being used within our project.
Is environment lightning just ambient light or environment map / HDR ?
Is area lights RectAreaLight exclusively?
And I’m sorry to repeat the question, but I don’t think I got a response, is ‘material model implemented in the project’ the modified PhysicalPathTracingMaterial?
We only use MeshPhysicalMaterial, MeshStandardMaterial and MeshBasicMaterial, so I’ll go check to replace the MeshBasic instances with MeshStandard, to check on whether that might be the problem.
So for now, this is as close as I can get to an error that looks similar to mine.
I copied the code on basic.js within the library and imported all the modules that this example is using. I’m getting black screen as soon as path tracing gets enabled. I’ll keep looking into trying to give a closer example on what we are actually doing to narrow down the problem.
Sandbox:
I’m not sure what’s going on with your codesandbox - but it seems like a build issue with “three” not being resolved:
You’re using various old webpack dependencies, as well, which I’m not familiar with. So unfortunately I can’t help you with build issues. Perhaps @drcmda has some ideas here.
Here’s a code sandbox using Parcel that works without web-worker bvh generation - I’m not sure why those are not working in CSB:
material model implemented in the project’ the modified PhysicalPathTracingMaterial
The “PhysicalPathTracingMateiral” is a custom fullscreen shader that implements all the logic needed for path tracing as well as material sampling for ray scattering when a triangle mesh is intersected.
Hi @gkjohnson, I’ve finally was able to make your library work within our React app! Thanks a lot for the help! I do have a question regarding the environment map showing around the scene. Is it posible to have path tracing applied to our scene GLB without the background showing? I tried commenting this piece of the script:
new RGBELoader()
.loadAsync(
"https://raw.githubusercontent.com/gkjohnson/3d-demo-data/master/hdri/hilly_terrain_01_1k.hdr"
)
.then( ( texture ) => {
texture.mapping = THREE.EquirectangularReflectionMapping;
// this.scene3D.background = texture;
// this.scene3D.environment = texture;
this.ptRenderer.material.envMapInfo.updateFrom( texture );
} ).then( () => {
this.generator = new PathTracingSceneGenerator();
const { bvh, textures, materials, lights } = this.generator.generate( this.scene3D );
const geometry = bvh.geometry;
this.ptMaterial.bvh.updateFrom( bvh );
But the background is still showing:
In the docs you can see the alpha flag on the renderer and the background opacity uniform for the material.
This works! Thanks a lot! This however gives me a black background. Any chance to get a white background?
I’m also running into a problem when adding a Reflector into the scene and then trying to apply path tracing to it. This Reflector does not belong to any GLB, its just added to the scene after created:
const reflector = new Reflector(
new THREE.PlaneGeometry( 61, 80 ),
{
textureWidth: REFLECTOR_RESOLUTION,
textureHeight: REFLECTOR_RESOLUTION,
encoding: THREE.sRGBEncoding,
recursive: 1,
multisample: 1
}
);
reflector.position.z -= 161; //161
reflector.position.y += 36.5;
reflector.position.x += 45;
this.scene3D.add( reflector );
Looks like a problem while accesing the object RGB values due to the ShaderMaterial composing Reflector not having any color property. Any workaround to this?
5[.WebGL-00006C3C03DF3800] GL_INVALID_OPERATION: Mismatch between texture format and sampler type (signed/unsigned/float/shadow).
localhost/:1 [.WebGL-00006C3C03DF3800] GL_INVALID_OPERATION: Mismatch between texture format and sampler type (signed/unsigned/float/shadow).
localhost/:1 [.WebGL-00006C3C03DF3800] GL_INVALID_OPERATION: Mismatch between texture format and sampler type (signed/unsigned/float/shadow).
localhost/:1 [.WebGL-00006C3C03DF3800] GL_INVALID_OPERATION: Mismatch between texture format and sampler type (signed/unsigned/float/shadow).
localhost/:1 [.WebGL-00006C3C03DF3800] GL_INVALID_OPERATION: Mismatch between texture format and sampler type (signed/unsigned/float/shadow).
localhost/:1 [.WebGL-00006C3C03DF3800] GL_INVALID_OPERATION: Mismatch between texture format and sampler type (signed/unsigned/float/shadow).
Pass.js:64 WebGL: INVALID_OPERATION: bindTexture: textures can not be used with multiple targets
Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'r')
at MaterialsTexture.updateFrom (MaterialsTexture.js:184:1)
at viewer3d.js:234:35
Thanks in advance.
renderer.setClearColor( 0xffffff, 1);
should work
you could feed it a color?
const reflector = new Reflector(
new THREE.PlaneGeometry( 61, 80 ),
{
textureWidth: REFLECTOR_RESOLUTION,
textureHeight: REFLECTOR_RESOLUTION,
color: 0x000000,
encoding: THREE.sRGBEncoding,
recursive: 1,
multisample: 1
}
);
i’m not sure where you saw reference to the encoding parameter you’ve set here but that doesn’t seem right, this seems like it would be set on the renderer…
the options inside the reflector class seem to be as follows…
{
color: THREE.Color,
textureWidth: Number,
textureHeight: Number,
clipBias: Number,
shader: THREE.ShaderMaterial,
multisample: Number,
}
setClearColor was not working for some reason, but found out that this property on FullScreenQuad was what I was looking for:
this.fsQuad = new FullScreenQuad( new MeshBasicMaterial( {
map: this.ptRenderer.target.texture,
blending: THREE.CustomBlending,
} ) );
The background problem is now solved.
I do however still have problems with the Reflector. Feeding a color does not seem to work for the Reflector, even though it now has a color property. I will have to debug further.
As to why I was using the encoding property within Reflector, I was following this issue due to past performance problems when adding Reflector.
Thanks for pointing me out to the right direction.
this meant adding encoding: THREE.sRGBEncoding to the “parameters” object inside the actual Reflector.js class… the issue was from 2019, the latest versions seem to have solved this in the onBeforeRender function inside the class…
Thank you very much for clarifying this, had some performance issues and was trying to make sure to discard any posible option.
After debugging my Reflector problem further, it seems like:
for ( let i = 0, l = materials.length; i < l; i++ ) {
const m = materials[ i ];
// sample 0
// color
floatArray[ index++ ] = m.color.r;
floatArray[ index++ ] = m.color.g;
floatArray[ index++ ] = m.color.b;
this part of the code is trying to acces the color property of the material, which is not finding (for some reason?) even if I add it to the parameters as @Lawrence3DPK suggested. When I add the Reflector, all the rendered objects turn black.
Added to this, I’m finding this artifact when I let the path tracing to work for a while:
I did some research and this looks a lot like what some people call ‘fireflies’. Any idea what might be causing this?
If you go to this demo and load up the millennium falcon model from the drop down, you’ll see reflections on the floor, it’s probably best to follow suit on the approach in this official demo for reflections while using three-gpu-pathtracing, I’m not at my machine atm but it’s potentially using ssr
