Excessive memory usage for GLTF file

[ericman314]

I have a ~5 MB GLTF file that is consuming about 9 GB of memory during and after loading. The file has no textures, just meshes. This 1800x increase in size seems a bit excessive, and I’m wondering if this is due to something goofy about the GLTF file itself, or if it could be a bug in the GLTF loader, or if this is just the way it has to be because of WebGL reasons.

oval.gltf (4.6 MB)

@gltf-transform inspect (see below) says that the file has just a single mesh of size 1.23 MB, with 9928 primitives and 386,437,472 vertices. I’m assuming that the majority of those vertices are reused in order for the mesh to only require 1.23 MB.

But it looks like the GLTF loader is creating about 9 GB of array buffers:

Inspecting the resulting scene shows a group of 9928 meshes, as expected:

And here is one of the geometries of those meshes:

What’s notable is that every single one of the meshes has identical position and normal arrays. This makes sense, as those arrays should just mirror the buffer in the gltf file. (This being a BufferedGeometry, each mesh should just consume the array indexes that it is interested in.) And so I would expect each geometry to point to the same FloatArray32 reference. At first glance, this appears to be the case:

However, for some reason, starting with index 317 a new array is allocated for each mesh:

And this continues all the way to the last mesh. This is very suspicious, and looks like a bug, but I can’t be sure without understanding how GLTFLoader.js works a lot better than I do now. I wanted to post this question here first to see if there may be something obvious that I’m just not aware of as to why the loader is using so much memory and copying the arrays instead of reusing them.

Here’s the output from @gltf-transform inspect:

 OVERVIEW
 ────────────────────────────────────────────
┌────────────────────┬───────────────────────────────────────────────────────────────────────────┐
│ key                │ value                                                                     │
├────────────────────┼───────────────────────────────────────────────────────────────────────────┤
│ version            │ 2.0                                                                       │
├────────────────────┼───────────────────────────────────────────────────────────────────────────┤
│ generator          │                                                                           │
├────────────────────┼───────────────────────────────────────────────────────────────────────────┤
│ extensionsUsed     │ KHR_texture_transform, KHR_materials_specular, KHR_materials_transmission │
├────────────────────┼───────────────────────────────────────────────────────────────────────────┤
│ extensionsRequired │ none                                                                      │
└────────────────────┴───────────────────────────────────────────────────────────────────────────┘



 SCENES
 ────────────────────────────────────────────
┌───┬──────┬──────────┬───────────────────────────┬────────────────────────┐
│ # │ name │ rootName │ bboxMin                   │ bboxMax                │
├───┼──────┼──────────┼───────────────────────────┼────────────────────────┤
│ 0 │      │          │ -0.0762, -3.9624, -0.0508 │ 0.0762, 3.9624, 0.0508 │
└───┴──────┴──────────┴───────────────────────────┴────────────────────────┘



 MESHES
 ────────────────────────────────────────────
┌───┬──────┬─────────────────────────────────────────┬────────────┬──────────────┬─────────────┬─────────┬──────────────────────────┬───────────┬─────────┐
│ # │ name │ mode                                    │ primitives │ glPrimitives │ vertices    │ indices │ attributes               │ instances │ size¹   │
├───┼──────┼─────────────────────────────────────────┼────────────┼──────────────┼─────────────┼─────────┼──────────────────────────┼───────────┼─────────┤
│ 0 │      │ TRIANGLES, TRIANGLE_STRIP, TRIANGLE_FAN │ 9,928      │ 374,084,682  │ 386,437,472 │ u32     │ NORMAL:f32, POSITION:f32 │ 1         │ 1.23 MB │
└───┴──────┴─────────────────────────────────────────┴────────────┴──────────────┴─────────────┴─────────┴──────────────────────────┴───────────┴─────────┘

¹ size estimates GPU memory required by a mesh, in isolation. If accessors are
  shared by other mesh primitives, but the meshes themselves are not reused, then
  the sum of all mesh sizes will overestimate the asset's total size. See "dedup".



 MATERIALS
 ────────────────────────────────────────────
┌───┬──────┬───────────┬──────────┬───────────┬─────────────┐
│ # │ name │ instances │ textures │ alphaMode │ doubleSided │
├───┼──────┼───────────┼──────────┼───────────┼─────────────┤
│ 0 │      │ 9,928     │          │ OPAQUE    │ ✓           │
└───┴──────┴───────────┴──────────┴───────────┴─────────────┘



 TEXTURES
 ────────────────────────────────────────────
No textures found.


 ANIMATIONS
 ────────────────────────────────────────────
No animations found.

[donmccurdy]

Two likely issues:

1. three.js cannot currently share buffers across multiple geometries, which would be required to optimize this file. See Improving BufferAttribute (maybe) · Issue #17089 · mrdoob/three.js · GitHub for the relevant proposal.
2. the file does seem a little suspicious to me, and 10K primitives is a lot regardless of buffer reuse

Not sure which of these is more relevant to what you’re trying to do, though.

[ericman314]

Ah. So despite each geometry only using some of the buffer, each geometry needs its own copy. That makes sense.

I wonder if the GLTF file was just a single mesh instead of nearly 10000 meshes, that would solve this issue. Unfortunately, I don’t control the generation of the GLTF file, but it’s possible I could do some post processing on it to combine the meshes. I tried gltfpack but it failed with null function or function signature mismatch for this file. I will have to try something else.

[donmccurdy]

Hm, gltfpack worked for me on this file! Try:

gltfpack -i oval.gltf -o oval_packed.glb -noq

The result is much more workable: 1 draw call, 40,000 vertices, and 1.2 MB.

[ericman314]

I got gltfpack to work by using the pre-built binary rather than through npm install. I have also successfully used gltf-transform join. The resulting file is much more manageable and loads very quickly in three.js. But both gltfpack and gltf-transform use just as much memory to optimize oval.gltf, putting me right back where I started. I’m processing these files automatically in response to a user upload, so all optimizing will do is move the memory problem to a different point in the process.

It looks like I’m going to need to reach out to the vendor of the software that I’m using to create the gltf files, and see what they can do about not writing such a wildly inefficient file. Thanks for your help.

[donmccurdy]

Yes, if the vendor could produce files with fewer mesh primitives, that would be much better.

If you’re accepting user uploads through a webpage, it would also be possible to run the gltf-transform processing there on the client device before upload, but I don’t know if that’s practically useful, maybe you want to keep the original upload anyway as a safeguard.

Do you mind if I post oval.gltf to a bug on a GitHub issue for glTF Transform? It isn’t currently handling the triangle strip and triangle fan primitives correctly (these are rare) and I’d like to work on a fix for that. See Unable join() primitives with TRIANGLE_STRIP or TRIANGLE_FAN modes · Issue #1312 · donmccurdy/glTF-Transform · GitHub.

[donmccurdy]

If the vendor has the option to produce a .glb file, that alone will reduce size by ~33%, but won’t affect the performance issues. If they don’t have that option, I’d encourage them to add it, for the reasons described in Don't use Data URLs in 3D models.

[ericman314]

Sure! You can post it. A fix for the triangle strip and fan would be appreciated by many, I’m sure.

[ericman314]

How familiar are you with three.js’s GLTF loader? I found this snippet in BufferGeometryUtils.js which converts triangle strips/fans to regular triangles. It clones the geometry and does some reindexing:

    // build final geometry

    const newGeometry = geometry.clone()
    newGeometry.setIndex(newIndices)
    newGeometry.clearGroups()

I have a hunch that geometry.clone() is responsible for the excessive memory usage, since in oval.gltf, there are 317 regular triangle meshes, and the rest were fans and strips, and it was only at that point that new Float32Arrays were being allocated. If it could somehow reuse the Float32Array but still clone the other properties of the geometry, I bet that would work.

[ericman314]

Here’s my solution:

    // build final geometry

    const newGeometry = geometry.clone()
    newGeometry.setIndex(newIndices)
    newGeometry.clearGroups()

    // Replace each attribute's array with the original array, and let GC clean up the new arrays that were just created by .clone()
    for (const key in newGeometry.attributes) {
      newGeometry.attributes[key].array = geometry.attributes[key].array // use original array
    }

I have no idea if this is safe to do, but it works correctly in my case. It reuses the original geometry’s array buffers, and allows the GC to clean up the arrays that were allocated for the cloned geometry. It doesn’t improve performance or reduce the number of draw calls, but at least it no longer crashes the browser/server.

I’m happy to raise this issue on GH if you think that would be helpful. I realize that my “solution” is not suitable for everyone as-is, but perhaps it could highlight the problem and lead to a better solution.

[donmccurdy]

In this input mesh, each geometry shares the same two vertex attributes. And then each of the 10,000 geometries has different indexes that point to a few vertices from those vertex attributes. three.js is going to upload each of those geometries to the GPU separately, without sharing the buffer, as far as I’m aware.

The geometry.clone() calls are certainly not helping matters either, and I think it’d be fine to file an issue on GitHub about that. But I wouldn’t want to make promises about how much we can improve performance on this one, short of Improving BufferAttribute (maybe) · Issue #17089 · mrdoob/three.js · GitHub, which is quite complex…, so you may want to have other workarounds available.

[ericman314]

Ah, that does make sense. I noticed that although the memory consumption is much lower when run in a regular browser, it still uses a lot of memory when run in a headless browser, probably because it is not using the GPU/hardware acceleration, and, as you said, it’s still sending a copy of each array buffer to the “GPU” which for a headless browser, is just RAM.

I will continue looking for workarounds, but this hack alone should save 50% of the memory usage (close to 100% on the browser side, but 0% on the GPU side), and that’s a lot better than nothing. Thanks for all your help!

[ericman314]

I opened an issue. Don't duplicate array buffers when loading GLTF triangle fans and strips · Issue #27926 · mrdoob/three.js · GitHub