- It depends on the amount of these meshes / instances - but doing keyframe calculations on the CPU may be less performant than just not doing instancing altogether (without instancing you’ll also be able to optimise animations with LOD levels and updating only the objects close enough to the camera.)
- If nevertheless your scene is indeed packed with billions and billions of these meshes - this fork may be just what you’re looking for. It’s based on this discussion - wasn’t merged to the core though, you’ll need to add it manually.
- There’s also this - but it’s WebGPU only and not super widely supported at the moment.
1 Like
-
I would do all the calculations once up front and bake them into a texture. Eg 20 frames x Vector3 = 90 Height X 2000 Vertices Width
-
I might try that branch out. A typical maxed out scene would be ~400 units per player, at < 2000 triangles a unit. Up to 8 players. In addition to the terrain (50k triangles divided by 8x8 geometries for culling/LOD) and other about 150-200 small items on the map (“non player units”). The camera is typically looking at about 1/8 to 1/4 of the entire map. In total there are about ~100 unit types, so 100 unique textures / meshes + 64 textures/meshes for the terrain. This is all discounting particles/overlays which is another thing altogether.
-
I need to round out my understanding of what I need to be concerned about when using instancing.
b) I guess I cannot use LOD with instancing?
c) How do I efficiently omit drawing without relying on “count” as I don’t know if I can use that without reshuffling everything.
There is a difference between instances being individually animated and all instances sharing the same animation state which would come by default if not extended to an texture atlas.
You can more or less easily expand the current system for instancing, essentially you will need skeletons sharing the matrices buffer and all be packed into it, writing into it with their offset they been assigned to as well as the shader reading from it. Essentially a texture atlas for the bone texture.
Since you need a animation mixer for animation it would be more of a entirely additional task to move the animation processing onto GPU, i just throttle and reduce animations based on distance and other, it should totally suffice. ( you can also do by mixing 2 frames and linear lerp between their matrices, a bit more work but will still animated smooth with little detail loss)
You don’t have LOD regarding geometry unless you use one instance stack per LOD.
You can limit the amount of instances dynamically, just the buffer should be fixed. You could use different techniques to expand that or chunk it if needed.
You wouldn’t happen to have an example of this would you? I will need to animate individually each unit. Anyways, long journey ahead for me.
Well no, but yesterday evening i gave it a more deep thought and i will make a plugin for it soon with a simple to use API so your actual mesh object can stay in the scene as usual.
The point is that LOD can become a must, less regarding geometry, more regarding the bone texture. The more bones per mesh the larger the bone texture atlas becomes, assuming your character is detailed with fingers and facial bone rig, going with round about 100 bones, 100 meshes would already take up a 400x400 32bit texture that is uploaded every frame.
So my idea is to LOD the bones as well just as i already skip their computation, you will set a priority on bones so the lower priority ones in a lower LOD will not be included, resulting in a smaller bone texture - essentially only closer characters will require the highest LOD where the texture can be updated the partially to the used amount.
It should be enough for most causes to throttle and reduce animations by LOD, to max this out animations could be threaded, but that’s a bit more complex and more work and the question is if it’s necessary, in the end the main constraint remains the upload to GPU of the transformations. But depending on the asset/bone count it should be possible to get several hundreds to several thousand for a real world game with the LOD system, but using a spatial index obviously would give it more juice as for a regular scene the plugin will just as the renderer process things linear and without culling.
Implemented it so far, instanced skinned mesh with individual animation states, also with the LOD system in-built to not only use a reduced geometry but also reduced set of bones.
The strategy is that instances are usually not all in front of the camera, so most likely fall into the lower LOD depending on the distance you set for them, generally it’s about keeping uploads of bone matrices small as possible asides of reducing animation computations.
Still need to test through the LOD part further and see how far i can reduce animations without core modifications, but they could be already throttled by the LOD using the onUpdate callback to update your animation mixer.
The general idea is that each instance only uses 1 Mixer for each LOD, so the reduced bones are remapping the skin weights to parents for those bones not used in the LOD. Otherwise you would obviously need 3 Mixer for 3 LODs.
The other benefit is you can attach things and items as you would usually regardless of the LOD. And you can unlink it from the instancer anytime so it will render as regular SkinnedMesh again.
So far the API looks like (for 1 LOD)
const mixers = new WeakMap;
// Creates a instancer for this type of asset, you don't use the original of the glTF in the scene but make clones from it
instancer = new SkinnedInstancer({
target: playerMesh,
maxCount: 1024,
levels: [
{
distance: Infinity
}
],
onUpdate: ( instance, delta ) => mixers.get( instance ).update( delta )
});
scene.add( instancer );
// Clones the mesh and it's skeleton appropriately and
// calls instancer.link( mesh ) to render it through the instancer, but you
// still add and use the clone as usually in the scene.
const player = instancer.cloneLink( playerMesh );
mixer = new THREE.AnimationMixer( player );
mixers.set( player, mixer );
scene.add( player );
5 Likes
That’s mighty impressive. I hope to get my hands on that! 
After playing around a bit I’m realizing my original implementation was assuming I could have an instanced attribute per vertex but I’m finding out I cannot! At most I can have a matrix4 (instanced buffer attribute) per instance, which definitely ups the challenge. One of my challenges is to keep the interface as uniform as possible because I’m not only supporting 3d models but the classic 2d graphics (and in fact they can change depending on active camera and can also coexist!). So I have an image pool, and it’s responsible for handing out 3d objects, which now will be more like proxies (instances) even if they are actual individual meshes in some cases. I think that would tie in nicely to your API.
I have looked partially into spatial indexing but have not needed it yet. I see the benefit but at a non-trivial complexity cost!
That LOD system sounds pretty nuts. I would imagine that is quite complex algo-magic and above my head. I was originally just planning on doing 2 LOD levels manually and hadn’t considered yet what to do with animations. The only other thing I can think of is the integration with three materials since I do some team colors modifications, emissive intensity and such.
1 Like
So automatic bone reduction works now, but the mesh should not have errors like the Soldier.glb in the three examples, it has skin indices pointing to bones it does not has.
Anyway with any other models i tested there seems to be no issue, the only thing left to do is figuring out why THREE on the render call still is doing some matrix computations, and unfortunately i don’t have a asset with multiple geometry LODs here to test i could also use for a demo, but there should be no issue as long as every LOD is assigned to the same skeleton.
It also skips processing animation bindings or matrix computations on inactive bones. With debug mode enabled it will visualize the LOD levels.
Max instances (per LOD) 256 (so 512 in total)
LOD 0 texture 32x32, bone count 67, texture 1 MB
LOD 1 texture 8x8 ,bone count 14, texture 0.0625 MB
The second LOD basically skips all bones from hand on and form foot bone on
You see fingers not being closed or feet bending
All bones active
To properly test and showcase i need some multi-geometry asset, i can’t use those from my project unfortunately.
5 Likes
Holy s***. That’s some seriously impressive stuff! Well done!
1 Like
Very impressive. Two questions:
- are you planning to make this available to use in the near future?
- is it possible to control the animation state individually / instance? It’s fine if it has to be the same animation, but would it at least be possible the control the current time and/or perhaps the speed?
Yes i wanted to release it last weekend but was too busy, also still looking for better assets for better testing.
Yes you can also use different animations, just like a regular SkinnedMesh.
3 Likes
Sounds promising. Can’t wait…
@Fyrestar this is amazing, will this be an external jsm “addon” module or are you planning to make a pr on the main three.js library?! Exciting stuff! Can’t wait to see it in action!!
1 Like
Have you by any chance made any progress on this? I’m really looking forward to trying it out, it looks awesome.
1 Like
Yeah i just been still looking for some demo asset. Might make one for it, needs multiple geometry LODs.
1 Like
I put together a user-land port of https://github.com/mrdoob/three.js/pull/22667 in https://codesandbox.io/s/2yfgiu (also left a comment on the PR).
2 Likes
Hello…i was wondering if you ever released the instanced skinned meshes code for use in the threejs community. I found your post from 22 and really would like to apply what you did.
Not yet sorry, i was very busy with work and currently still am, but i release it within next 1-3 weeks, even though it’s technically done there is some adaption work to do to make it work across many Three releases, and making the demo asset. Originally i wanted to use a high-res human with many LODs but C4D doesn’t properly export animations in FBX (i use a custom asset format personally) so i’ll just see to get some basic multi-lod model done for the demo.
1 Like
If it helps, i can provide you with a high density multi-LOD fbx file to help in your development. To what email should i send it?
Hello, Is there any update on this topic? I’m curious if there have been any recent developments. Thank you.
2 Likes