How would one go about creating a loading behaviour similar to how Sketchfab loads 3d models and textures dynamically. i.e. once the geometry is loaded, it is added to the scene… then the textures are streamed in increasing levels of resolution.
Here’s an example loaded with sketchfab:
The same model loaded with three.js gltf loader looks like this:
You can see that the model is only displayed once all the loading has been finished.
Is this simply a matter of storing low resolution versions of the textures so that the initial load time is shorter? or is there some fancier way of streaming the texture data?
I’m working with gltf files, some are binary, and some are JSON with either embedded or external texture/bin files. All files are hosted in external storage and accessed via http (as opposed to being stored locally on the machine).
Almost any time you have a texture on a model, there is a mipmap chain with smaller and smaller versions of each texture. For example:
1024x1024
512x512
256x256
...
2x2
1x1
With PNG and JPEG textures (which most models use) the mipmap chain is usually generated on the fly when you load the model, resizing from the original high resolution image. With more advanced GPU compressed textures (like DDS, KTX, and Basis) it isn’t possible to generate mipmaps on the fly, so you have to precompute them all instead. They’re sort of inconvenient, but these textures have some performance benefits, so a viewer like Sketchfab is probably using them. The other advantage of this is, if you’ve precomputed a bunch of low-resolution versions of your image that you need to download anyway, you might as well download the low-resolution versions first, so you can start adding them to the scene and increase resolution later. I expect that’s what Sketchfab does.
You could do the same thing with PNG/JPG textures, but in practice people usually don’t. I’m not sure if there’s an important reason for that, or if it’s just inconvenient. In any case your model already contains the full-resolution version of the image, so if you’ve already downloaded that, you might as well render it.
glTF will be adding support for GPU compressed textures with streaming capabilities, and the beginnings of that are in https://github.com/mrdoob/three.js/pull/18490. But the streaming part isn’t there, yet, so there is a ways to go.
In the meantime you’d need to manage your textures yourself, outside of the model, to attempt this.
6 Likes
Great explanation, thanks.
So the mip-mapping is almost what I want… except it only happens after load time, so not very useful for what I want to achieve… as you’ve pointed out - I already have the full resolution texture at that point.
Would the following work in theory?
- Store several versions of each texture with decreasing size (for example 1024, 512, 256, 128).
- ensure that the model is referencing the lowest resolution texture
- After loading the model, use some other logic to download and replace the low resolution texture with the next highest resolution version… repeating until the highest resolution has been loaded
Functionally that will work, yes. You don’t necessarily need to store the whole chain, a couple files might be enough if you generate mips on the fly from the final high-res image at the end. Practically there’s one other obstacle with PNG/JPEG images, which is that larger images take a while to decode and upload uncompressed to the GPU. The result is that, after you’ve already started rendering with the low-res version, you might have XXms of freeze when the high-res image is uploaded to the GPU. This might be OK depending on your use case. You can profile the application and see this with your existing model — the texture GPU upload calls are often easy to see in the Chrome dev tools profiler.
1 Like
OK, thanks. Good to know that I can use the dev tools to profile this.
I can confirm that loading low resolution textures indeed has a huge impact on initial load of the model. On my (relatively high performance) machine, loading 512x512 resolution textures for a simple model results in instant display, increasing texture resolution to 1024x1024 results in 3-4 seconds load time delay.
I believe something like simple progressive texture image loader for Three.js · GitHub can help achieve what we want.
See this comment about rendering two scenes in the background, and alternating between them while one loads a texture, and the in a live example(s) further down. The concepts is basically you may be able to have two canvases, and two workers with OffscreenCanvas that you alternate between (only one of the scenes is loading a texture at a given time, and the other does rendering), to give the appearance that there is only one scene that does not pause when textures are loaded.
1 Like
