InstancedMesh2 - Easy handling and frustum culling

Discussion
1

Library posted here:

…

Hi, I have been working on an extension to InstancedMesh for a couple of days now.

InstancedMesh2

I have created my own InstancedMesh2 class that extends InstancedMesh, which manages an array of InstancedEntity, itself a class that is similar to Object3D that allows you to manage the position, scale, rotation and visibility.
Visibility is managed by swapping the element you wish to hide with the last one in instanceMatrix array, and decreasing the count by 1.
This allows the hidden element not to actually be rendered.

This approach uses more memory but:

  1. it simplifies object transformations
  2. it allows hiding all instances that are not in the camera frustum (I am working on this with good results, using a custom octree and other optimisations, even if i want to try BVH)
  3. it allows the easy creation of an InstancedLOD (which I am working on, also with good results)

I tried using an InstancedMesh2 with a count of 2kk, using low poly geometry.
In a favourable scenario, with static objects and camera, thanks to the frustum control, it was possible to hide 85% of the instances. Frustum check needs to be done after each camera transformation (still considering static objects).

image
image1006×994 320 KB

How to use it

const monkeys = new InstancedMesh2(geometry, material, count, {
  behaviour: CullingDynamic,
  onInstanceCreation: (obj, index) => {
    obj.position.random();
    obj.scale.setScalar(2);
    obj.quaternion.random();
  }
});

scene.add(monkeys);

// How to handle instances
monkeys.instances[0].visible = false;
monkeys.instances[1].rotateOnWorldAxis(xAxis, Math.PI);
monkeys.instances[1].updateMatrix();
monkeys.instances[2].position.x += 10;
monkeys.instances[2].scale.multiplyScalar(2);
monkeys.instances[2].updateMatrix();

Live Example (no frustum check)

I will make all my work public and usable by anyone, as packages with only three.js as peer dependency, and would love to receive advice.

I thank @DolphinIQ for the advice :slight_smile:

Currently a similar version without visibility management, but with interaction events is already available in my three.ez library.

In future posts I will explain how I plan to optimise visibility management according to the frustum of the camera and perhaps how I would like to manage InstancedLOD.

Update:

Last example (1 million instances + frustum culling)

https://stackblitz.com/edit/three-ez-instancedmesh2-cullingstatic-1kk-forest?file=src%2Fmain.ts

Package released

7 Likes
2

Would this change the order of meshes?

  • Apart from the matrices, other instance data should also be changed (e.g. colors and any custom attributes).
  • In some cases I have instanced mesh accompanied by a parallel array of additional non-graphical instance data (mapped 1-to-1 to the instances data). Exchanging instances would break this mapping.
2 Likes
3

Great observations, as usual :slight_smile:

No, the instances array will remain the same.
There is an internal, invisible to the user, handling of the instanceMatrix and instanceColor indexes.

instancedMesh.instances[0].visible = false;
// instancedMesh.instances[0] is still the same element.

Both matrices and colors (if present) are currently swapped.
Custom attributes may be swapped with a utility.

This is even simpler.
You don’t need an array, but to add the data to the instance.

class Entity extends InstancedEntity {
  customData = 10;

  constructor(parent: InstancedMesh2, index: number) {
    super(parent, index);
    // set position, scale, quaternion.
  }
}

const myInstancedMesh = new InstancedMesh2(geometry, material, 1000, Entity);
myInstancedMesh.instances[0].customData = 15;

Do you think something can be improved?

1 Like
4

Hm. Does it mean that matrices and colors are swapped internally, but the user must swap all other THREE.InstancedBufferAttribute arrays?

Does it have cloning? (De-)Serialization?

Any plans to make BatchedMesh2 over the expected BatchedMesh?

1 Like
5

You are right. It should be possible to swap also InstancedBufferAttributes without the user doing it manually.

However, geometry can no longer be shared (only if has custom attributes) and InstancedBufferAttributes with meshPerAttribute greater than 1 would not be compatible.

Right, they are methods that must be there, but I will include them at the end.

I’m waiting for r159, but if you think it would be useful, I’ll gladly do it.

I saw this issue on BatchedMesh and I have the impression that we are doing similar work… I’d love to talk to him.
InstancedMesh2 already allows you to share entities with other InstancedMesh2 (useful for InstancedLOD, but needs improvements).

  // this is still working in progress, will be easier to use
  const lod = new InstancedLOD(count)
  lod.addLevel(new THREE.IcosahedronGeometry(1.5, 16), material, 0.5);
  lod.addLevel(new THREE.IcosahedronGeometry(1.5, 8), material, 3);
  lod.addLevel(new THREE.IcosahedronGeometry(1.5, 4), material, 10);
  lod.addLevel(new THREE.IcosahedronGeometry(1.5, 2), material, 20);
  lod.addLevel(new THREE.IcosahedronGeometry(1.5, 1), material, 50);
 
  for (let i = 0; i < lod.count; i++) {
    lod._sharedData[i].position.random().multiplyScalar(1000).subScalar(500);
    lod.updateInstancedMatrix(i);
  }
2 Likes
6

Small experiment with InstancedLOD and 2kk meshes.
This is also a favourable scenario, where 88% of the meshes were hidden thanks to the frustum control
The goal is to try to have similar results by calling up a continuous update of distance and visibility calculation, although I don’t think it will be possible, even if @gkjohnson made me believe in magic with three-mesh-bvh :smiling_face_with_three_hearts:

image
image1516×1006 247 KB

Little memory problem:
image

1 Like
7

I have prepared an example where it automatically swaps InstancedBufferAttribute.
In addition, I added the possibility of passing a callback executed for all instances.

const monkeys = new InstancedMesh2(geometry, material, 20000, (obj, index) => {
    obj.position.x += index * 5 - 20;
    obj.scale.setScalar(2);
    obj.updateMatrix();
});

image
image724×448 86 KB

Thank you very much for the advice :smiley:

2 Likes
8

Small update:

I wrote an algorithm to minimise the swaps made in the array to manage visibility, and implemented a first test of frustum culling.

Can you please tell me if this example (50k animated meshes) runs at 60 fps on your computers?

image
image1920×914 389 KB

2 Likes
9

Oscillates in the range of 60-62 fps.

image

1 Like
10

Thank you :slight_smile: Do you think this is an acceptable result? (I don’t know your monitor refresh rate :joy:)

Currently a check is made on all meshes if they are inside the frustum (like BatchedMesh i think).
This is only useful if all meshes are animated every frame, otherwise there will be a flag indicating that the scene is static and in that case I can optimise a lot thanks to a hierarchical control.

And it will also be possible to optimise this example with an InstancedLOD ofc.

1 Like
11

The result looks nice, however … I don’t know what is acceptable. For every person it means something different (my card can reach 144 fps for simple scenes, and 60 fps for your demo).

Such performance tests are good for developers, but are not indicative for the end user … I mean, I have had many cases, when something looks fast on a demo page, but performance is notably worse when applied on a real page (with other objects, various materials, animations, effects, etc).

Edit:

Here is something strange. When I fly towards the most distant Suzanne, the others go outside the frustum. I expect this should increase the fps if frustum culling is active. But even when I end up with just one Suzanne, the fps is barely increased by 10% to 66 fps.

image
image1918×1042 78.5 KB

2 Likes
12

Probably in this case the slowness is caused by animating 50k meshes every frame.

Here is an example without animation, this one I hope doesn’t go below 144…

13

Yes!

image
image988×358 40.8 KB

1 Like
14

Well this looks like a good result, thank you very much for your help! :star_struck:

In a few days I hope to publish some examples of the InstancedLOD class.

1 Like
15

image

165 (my refresh rate) at 100.000 (animated) items on an RTX 2080ti.

1 Like
16

Thanks for the test :slight_smile:
I think you also have a high-end cpu to generate all those frames.
Updating 100k matrices for 165 frames is a heavy load for the cpu…

…

But it doesn’t really make sense to update the matrices of hidden objects… just like three.js does… I really need to improve this…

1 Like
17

Oh right, the CPU may have a role to play here. I’m running on an Intel I7 13700K.

1 Like
18

solid 144 fps on my gtx1080 when camera is in the center, and ~120 when zoomed out…

1 Like
19

Small update:

I optimised the update of the matrices… now only the visible instances matrices are updated.

@PavelBoytchev Now you will probably make more fps even with animated instances :slight_smile:

4 Likes
20

After a break, I resumed work on this class.

I implemented optimisations on static instancedMesh (without animations) using a bvh and Hierarchical Culling.
There are still things to improve but it is a good start.

Without a LOD the gpu works too hard, so that will be the next step.

Can someone test it and tell me the fps, to see if there is any improvement compared to the last example? Thank you.