What’s up everyone,
I’m making an art gallery using react-three-fiber. The gallery has 30 image textures in it and on mobile the textures take up too much vram, so I’m trying to compress them using either ktx or basis. I’m using the useLoader hook from r3f and my code currently looks like this but the issue is I’m only seeing black textures.
The texture in the third image is a .ktx texture and the console gives me the following warning “only compressed formats currently supported”
I’m compressing my textures using this repo
I looked over the documentation here Khronos Texture Tools: toktx but all of the compression methods they have produce a .ktx2 file instead of a .ktx file.
This has lead to a whole other problem entirely b/c though I can load ktx2 files on desktop with improved performance, they take way too long to load on mobile. I’ve also made sure to resize the textures to be powers of 2, which was what I had wrong at first with .ktx2 files.
I’m still new to this and I’m not really sure which problem I should be trying to tackle. Thanks for taking the time to read this and any pointers are welcome
.ktx2 files are probably what you want to use; .basis would be OK too but is functionally the same thing, in a somewhat less future-proof container. The older .ktx format doesn’t support Basis cross-platform compression and you’ll need different textures for different types of devices, I’d try to avoid that.
If you have a .ktx2 file I’d start by testing it on https://sandbox.babylonjs.com/ – if it works there that’s a good sign that the file itself is fine. You may also need to enable the toktx --genmipmap option depending on the filter options you’re setting on this texture.
Alright, thanks for the tip. I’ve got .ktx2 working properly on desktop, it’s just on mobile where things get really slow. I’ll try this with the genmipmap option. If I’m still having an issue I’ll post a temporary vercel link with the code as well.
Okay so here’s how I’m compressing the images as well as a link to the full project.
https://muse-place-ifbrdkboj-metaplug.vercel.app/homocosmico
I’ve done enough testing on the performance of this site to be certain that the image textures are the problem with the bad mobile performance. In this case the textures are loading in so slowly that they aren’t even there when the rest of the site loads. Here is also how I’m loading in the ktx2 textures
On a side note, I’m curious if there’s a way to store extra metadata on the ktx2 files b/c I’d like to store the original dimensions of the images so that I can rescale them on the threejs side, though I’m worried that rescaling them might interfere with the mipmaps
To clarify here —
- The issue is that the textures are loading slowly, not that they’re dropping framerate after loading, correct?
- Can you get the value of
texture.format, on one of the mobile devices where performance is a problem? If you can get a Chrome Performance Profile of the loading process that’s also very helpful but admittedly more work…
I’m guessing that the textures are just transcoding more slowly on your mobile devices, but it could definitely be a number of other things… if you can share the images and types of devices you’re testing that’d be interesting as well…
I’m curious if there’s a way to store extra metadata on the ktx2 files b/c I’d like to store the original dimensions of the images
Yes, but I don’t think toktx does anything like this automatically. If you don’t mind writing a script for it you can use ktx-parse. Something like this:
import * as fs from 'fs';
import { read, write } from 'ktx-parse';
const container = read(fs.readFileSync('input.ktx2'));
container.keyValue['originalSize'] = 'whatever';
fs.writeFileSync('output.ktx2', write(container));
But you’d also need a small patch in KTX2Loader to expose that data. It might be easier just to keep a JSON manifest separately.
I’m actually not certain if it is a loading issue or they’re dropping frames. I’m not sure how I would test that. I remember once a couple of the images loaded on my phone after I had been on the page for a while, but I’m not sure if that means that it’s a loading problem. Upon checking the chrome performance tab on desktop, it seems that the program has to decode all of the .ktx2 images and it’s happening after the site has “loaded” and the user can enter the site. All the dark green squares at the bottom are labeled as “Image Decode”. I’m not sure how this would hurt mobile performance though, I don’t know enough about how it all works yet.
Texture.format: 37492, on desktop it’s 36492, I’m looking up what this means.
This part is interesting I didn’t think about this. My current workaround is to upload a .txt with the original dimensions, and then read from those files later
Just above the green “Image Decode” blocks, are you seeing texImage2D, compressedTexImage2D, or something else? Maybe also worth knowing how many images are involved here, and their dimensions and formats… from the parts I can read, those look like normal textures and not Basis/KTX textures (which shouldn’t be decoded).
Ok, that’s ETC2 on mobile and BPTC on desktop, both sound fine.
