I’m trying to do something difficult: a Ken-Burns style animation of one large texture (A) cross-fading into another (B) and so on. Both textures may be moving or rotating (which really makes any frame rate stutters quite obious). The biggest difficulty is that each texture has to be loaded in real-time : no pre-loading.
The goal is to have buttery-smooth animation with absolutely no jank.
- must support 4K textures
- must work on macOS (so Safari / Webkit)
- must support IE11
Here are some techniques I’ve tried:
- Using Image.decode() promise so that image decoding happens off the main thread: HTMLImageElement.decode() - Web APIs | MDN I think this is broken in Chrome (it seems like the image would decode a second time, for no reason) but may actually work fine in Safari/WebKit.
- pre-loading the texture into THREE before it’s actually rendered, e.g. not adding it to a scene graph until the GPU upload has finished.
- instead of using glTexImage2D(), using glTexSubImage2D() to load the image in strips
Results are pretty good, but I still see occasional jank - I’ve used the Safari and Chrome debuggers to investigate, and it looks as if some operations are always going to take more than 16msec.
With a 4K texture:
- glTexImage2D() can take > 50msec
- glTexSubImage2D() can take > 16msec even for a fairly small part of the image
- generateMipmap() can take > 50msec
Am I doing something wrong?
I’ve recently stumbled upon the issue of premultiplyAlpha. My understanding is that by default, WebGL wants non-premultiplyAlpha whereas the browser uses premultiplyAlpha (for ImageData, Canvas context, etc.)
Could this be why my texture uploads are so slow, because of the un-pre-multiply operation that’s happening during texture upload?
If I didn’t have to support IE11, there would be lots of options:
- use a WebWorker to un-premultiply the data in a background thread
- compress the data using one of the many Web2 image compression formats…