Issues with complex scene on iOS

Questions
,
1

Hi all,

I am building a web app which is giving me some headaches on iOS and I’m hoping someone might be able to offer some suggestions as to how I might get around the problems.

Unfortunately I can’t share any URLs because my web app is a R&D project for a client and the WIP versions are commercially sensitive. And because of its sheer complexity, creating a cut-down version for demonstrative purposes will not illustrate the issues I’m having. Hopefully my issues are generic enough that someone might be able to volunteer some ideas though!?

image
image1587×768 72.2 KB

My app loads two .GLB files - one is an animated skinnedmesh character with facial morphs and the other is the environment in which they reside. I have compressed the .GLB files and the texture maps as much as I dare and the total payload for my biggest scene weighs in at around 30MB. My app features a cartoon character that the user can chat to aloud using natural language. So I also have a bunch of audio files which are sound effects and static dialogue for my character (some of the things my character says never change, so it’s easier to pre-load these audios instead of waiting for a TTS API call every time). My audio files weigh in at around 11MB and these are loaded with the page so that they play back instantly when called upon.

I also pre-load (fetch) some JSON files which specify the camera, lighting, clipAction animation ranges etc. So on page load I’m loading about 45MB of assets.

My app works beautifully on Chrome, Firefox, Edge, Safari on MacOS and even an old Android phone I have. And it works on my iPhone 12 too. On my wife’s old iPhone 8, however, it loads over and over before giving me that hateful error

A problem repeatedly occurred with https://…

I’m 99% certain this is a problem with RAM. I’m using inspect to debug the app and I have a bunch of logs to the console. Using those logs, the page appears to crash just before the first call to

WebGLRenderer.render(myScene, myCamera);

Despite the fact that I’m precompiling the scene using

WebGLRenderer.compile(myScene, myCamera);

first and that (usually!) works okay. I’ve tried using smaller textures, the override material, removing shadows etc. and occasionally the page does load but it’s very rare and difficult to ascertain why.

I’ve tried using instruments on my wife’s Mac to look at the memory consumption but I don’t really know what that’s telling me (at some points it says MobileSafari is using 320% and 2.1GB of memory WTF!?) and I can’t find any help on that because instruments looks to be for xcode debugging really. And I’m not much of a Mac user anyway.

What I really need is the ability to “sniff” the amount of available RAM on iOS devices. If it’s less than, say, 120MB, then I’ll point to a folder of smaller texture files. If it’s less than, say, 60MB, I’ll turn off shadows. If it’s less than, say, 30MB, I’ll turn off antialiasing. And if it’s less than 16MB, I’ll simply alert the user that the page won’t work on their device. Anything to avoid the current experience which is to simply have the page crash over and over before telling the user a problem repeatedly occurred!

Does anyone have any experience with this problem or any suggestions please?

Thanks so much.

2

This does sound like it could be a RAM issue to me. You’ve mentioned that the .GLB files and their textures are around 30MB on disk, but have you estimated their RAM cost? Textures in PNG, JPEG, and similar traditional image formats will always have much higher memory cost than size on disk, and your geometry may also be larger in memory if you’re compression (like Draco). A quick way to get the estimate would be https://gltf.report/.

3

Thanks for the swift reply @donmccurdy.

I’m using .webp format for all my texture map images and I’m estimating their RAM requirements using

image.width * image.height * 4 * 1.33

I log this to the console when all my textures are (pre) loaded. It reports 267.7MB. Is that a lot? It doesn’t seem like it to me, and if I reduce their resolutions any further then things start to look pretty crappy in my browser (I started with gorgeous 4096 × 4096 textures and I’m now down to 512 × 512 in most places) :o(

I uploaded my character .GLB to gltf.report (thanks for the link) but I don’t really know what it’s telling me tbh:

image

One thing I didn’t mention: my .GLB file has a single timeline animation (12000 frames @ 60fps) which I cut into chunks for separate animations using THREE.AnimationUtils.subclip(). This creates large objects with large Float32Array objects inside. Could these be using large amounts of memory do you think?

4

Ok, I guess the textures are not in that GLB? Or things are spread across many GLBs? The “gpu: 5.8 MB” here is the number you’d want to look at. The fact that disk is higher than GPU is unusual, perhaps the animations are contributing to the higher disk size, and would also contribute to memory size. If you open the ‘inspect’ tab in the left sidebar it will show more details.

I have a vague recollection that older iOS devices might have VRAM limits in the browser as low as 500 MB, but I don’t really know how to check that limit for specific devices. The 16–120MB limits you mentioned above are unrealistically low for iOS devices, but the actual memory cost of your scene is probably in the high hundreds of MB so you may still need to optimize down.

I can’t tell you what to optimize from just a screenshot, but the output of the ‘inspect’ tab mentioned above might be more useful. If you have a lot of files you could get the same report in CSV with:

npm install --global @gltf-transform/cli

gltf-transform inspect model.glb --format csv
5

@donmccurdy the 16 - 120MB limits were just examples, I have no idea what limits I’ll need to set yet.

You’re correct, my .GLB files don’t contain any materials at all. I create these in JavaScript after the .GLB files and .webp files have been loaded, because they can be spec’d by the user in the JSON files.

I’ve found some info about the onboard RAM on particular iOS devices and although this doesn’t tell me the actual VRAM allocated for safari, it might be a good starting point for me. I can use navigator sniffing in JavaScript to try and ascertain the iOS device and go from there, I think.

Thanks again for all your help so far!