I have the following problem:
On initial load of my app I set everything on my scene as usual, taking care of compiling and initializing all necessary geometries/textures/shaders on advance using the tools Three.js provides that will be rendered later.
I have some BIG 8k textures that I really need to use, a couple of them, and when they are uploaded to the GPU each of them take a couple seconds with the GPU completely busy.
So while the GPU is busy no frames can be rendered, which can be several seconds (many in low-end hardware/ integrated GPUs), and that, even if suboptimal, is perfectly fine for me since I do it only on initial load of the app or very specific user actions.
Now, the problem is those operations ‘freeze’ the GPU, so no RAFs are fired, but the main CPU thread continues to run as it seems it’s not a synchronous operation. That means that after creating everything if I want to show my 3D world animating some of the elements the whole intro/animation is skipped as it ‘happened’ while the GPU was busy and unable to draw anything on screen.
Is there some reliable way (not an arbitrary timeout in the CPU thread please ) to at least know WHEN everything is ready and the GPU is fine to render what you need?
I’m still running it in my head and I was thinking about a hack similar to the code you posted.
Still, I REALLY don’t trust setInterval() and setTimeout() and try to avoid them like the plague. Even when I want to delay something, or do something recurrently I try to use some better solution like Gsap, which uses RAF internally and has been a reliable ally for many years .
In my experience browsers do some weird things with setInterval() and setTimeout(), they execute them in the very beginning or the very end of the frame time so you may end up with race conditions or undesired effects because of execution order, and when the tab is inactive they are still running (though not always respecting the time you set !).
So, taking your idea I would tweak it slightly so it only relies on RAF:
let gpuBusy = false;
const initHeavyMaterials = () => {
scene.add(mySuperHeavyTextureMaterialMesh);
gpuBusy = true;
renderer.render(scene, camera); // NOTE This causes the GPU texture upload
requestAnimationFrame( () => animate() ); // The GPU is already busy, so this will wait. If it doesn't then my solution wouldn't work...
};
animate() {
// In theory the first time this function is called will be only when the GPU is able to render a frame
// after everything in the scene is properly instantiated/compiled/uploaded to the GPU
if(gpuBusy){
gpuBusy = false;
// FIRE YOUR INTRO/START/COOL ANIMATIONS SOMEHOW HERE
}
// HERE GOES YOUR NORMAL RENDER LOOP CODE
requestAnimationFrame( () => animate() );
}
If you need to do it sometime after the initial load and when the rendering loop is already running then you would destroy/stop the RAF loop and start it again in a similar way.
It would be cool if someone can figure out a less ‘hacky’ or even more reliable method to do it, so I will keep the thread open to give a bit of time to other forum participants to bring their ideas.
It’s not really a hack. requestAnimationFrame is only run when GPU has time to render a frame - so it detects exactly what you’re looking for
Personally I wouldn’t do that - they are often useful, since as you mention, these are the only continuous loops that run when the tab is switched changed or window is minimalised. And you want the logic to be run. Logic / animation mixers should run in a setInterval loop (esp. if networking is involved), and rendering / camera updates should run in requestAnimationFrame.
I may be wrong but my experience with setInterval is pretty troublesome.
For example, one thing that I found out the ‘hard way’ in a recent project is that sometimes the code inside it gets queued when the tab is inactive, and then when you make it active again it tries to execute it all at the same time, like trying to catch up, lol.
I know it’s weird, but I’ve definitely experienced it when leaving a tab inactive for several hours.
Also, as far as I know:
Many modern browsers (such as Chrome and Firefox) throttle timers like setInterval() and setTimeout() when a tab is not active. In most cases, the interval or timeout may be significantly delayed if the tab is inactive, typically limiting it to one execution per second (or even longer)
So you have no guarantee that it’s going to execute when you told it to do so…
But yeah, if the only thing the code does is set the value of a boolean it may be fine
I will give a try to the proposed solutions and see what seems to work best and report back to the thread so people can benefit from it in the future.
The shader compile/link operations are carried out asynchronously to your Javascript in a separate GPU thread./process (even without parallel compile implemented), so those calls appear to complete quickly. (for Chrome at least).
However when you make any WebGL call that needs to see the result of the compile/link operations, that call waits for them to complete.
I guess that applies to any async operation in the GPU involving CPU/GPU communication, like also texture upload.
So if that’s true, which seems logic, manually calling one of those ‘cheap’ WebGL operations would provide us with a way to have some kind of ‘onReady’ utility in the Three.js lib, to hopefully do something like the following without having to rely on RAFs, and perform everything synchronously:
scene.add(mySuperHeavyTextureMaterialMesh);
renderer.render(scene, camera); // That will make the GPU very busy
// This onReady method could execute internally one of those WebGL operations
// that rely on the GPU to have finished its job and be ready,
// and then fire a callback
renderer.onReady( () => {
// Now the GPU is idle and ready to render, do your stuff
})
My WebGL knowledge is basic at best, so I don’t know if this is genius or just plain stupid.
If it could work I would gladly make a PR to try to add it to the lib, any feedback is super welcome .
There are a few causes for hitches in the render loop.
is when new textures/models are loaded, and decoded. This happens on the main js thread, and can steal/block the animation loop.
when models+textures are uploaded to the GPU… this also happens on the rendering thread (inside renderer.render) so can also block the animation thread.
when shaders are compiled, (happens inside renderer.render) the first time a new material is rendered.
There are different approaches to dealing with each of these.
for 1. you can avoid starting your rendering loop until all the initial assets are loaded…
(or you can go through some heroics trying to load/decode things in a webworker… r3f has some tools for assisting this). You can also avoid adding Every new thing to the scene immediately, and instead add a few per frame as to not bog down the frame processing everything at once.
for 2. and 3. There is a renderer.compile() method that can force all materials to compile manually… you can call this after all things are added.
And then as @mjurczyk described… calling renderer.render() explicitly can also help, though this will only cause upload/compile of objects that are visible in the camera frustum.
You’ll still incur a GPU upload if using imageBitmap as a texture I think. Caching it for canvas vs for WebGL context are 2 different operations. (i could be wrong)
Usually actually stalling the renderer happens because people are adding ALL their textures/meshes/materials in a single frame.
This is what I use when I have a lot of textures to load:
let loadingManager = new THREE.LoadingManager();
let RESOURCES_LOADED = false;
loadingManager.onLoad = function(){
console.log("loaded all resources");
RESOURCES_LOADED = true;
initAll(); // continue the initialization process
};
let txtrLoader = new THREE.TextureLoader(loadingManager);
let imagLoader = new THREE.ImageLoader(loadingManager);
let cubeLoader = new THREE.CubeTextureLoader(loadingManager);
let gltfLoader = new GLTFLoader(loadingManager);
let audioLoader = new THREE.AudioLoader(loadingManager);
Also for the render.render command, are we required to use the render.renderAsync?
I had a program that was not using the GPU until I inserted the Async.
@phil_crowther
I’ve never heard of renderAsync. Got any references on that?
renderer.render() should be complete when the function returns.
but care has to be taken when calling it, that all the objects that you want uploaded, be visible within the frustum of the camera. Perhaps you were rendering a first frame without all the relevant objects in the frustum?
I believe I got the idea from the three.js example webgpu_compute_particles.html. It is not used in all (or maybe even most) of the webgpu examples. But, in my case, the GPU did not start working until I used that form of the command. Obviously, I could be doing something wrong that made using Async necessary. Which is why I was asking.
I’m just saying that decoding an image is a separate process from uploading that image as a texture to the GPU. It’s possible that ImageBitmap somehow addresses both of these operations, but it wasn’t clear to me how that works without explicitly referencing the webGL context the image/texture is going to be used in. I’ll check out that video you linked.
I load and instantiate all I need.
Then I make sure everything needed is visible and in the camera frustrum.
With that I force a manual render of my scenes, but before that I mark a flag gpuBusy as true.
Doing that will make the gpu busy as it uploads all textures to the GPU, an async operation hard to track (Frame 1 following screenshot).
Then in my main render loop I have:
animate() {
// my pre-render calculations
renderer.render( scene, camera );
if(gpuBusy){
gpuBusy = false;
rendererReady.dispatch(); // A signal I dispatch so I can use it elsewhere in my code to fire my start call
performance.mark("GPU is ready") // The mark you will see in the attached image
}
requestAnimationFrame(() => { animate(); })
}
What happens is on first frame triggered by the manual render the GPU will start its work, but it’s async. If the GPU manages to pull out a frame and a RAF is fired then it will try to render again, will take a long time if the GPU is maxed-out, and after the render the gpuBusy flag is reset. At that moment you know you can do your stuff.
) to at least know WHEN everything is ready and the GPU is fine to render what you need?
.
!).
.