[idea] Transforming Geometries rather than Meshes and their children, in a render loop

geometry
resizing
mesh
scaling
scale

#1

To change the size of a geometry somewhere in a scene graph, without affecting the size of children in the subtree, I am doing something like this:

        mesh.geometry.dispose() // rid the old geometry
        mesh.geometry = new SomeGeometry( /* new size values here */ )

Resizing a geometry like this is expensive because it recreates vertices all over again, and the old geometry needs to be garbage collected.

It is possible to achieve the same effect using scale on the Mesh, which performs better, but has the unwated side-effect of also scaling children in the subtree.

Scale on a Geometry can currently only be set once and not in a loop, so we can’t get a performance boost using this feature.

If we could, the code would look like this:

        const scale = { /* scale values to mimic size change */ }
        mesh.geometry.scale.x = scale.x
        mesh.geometry.scale.y = scale.y
        mesh.geometry.scale.z = scale.z

If we scale a mesh instead:

        const scale = { /* scale values to mimic size change */ }
        mesh.scale.x = scale.x
        mesh.scale.y = scale.y
        mesh.scale.z = scale.z

then this affects all children, which is what I want to avoid.

It’d be great for this to work, so we can simply scale a Geometry inside a loop, but not scale an entire sub tree.



Sample Use Case

I’ve made some custom elements that render using CSS3D and WebGL (Three.js). They might remind you of A-Frame in some ways. I can do this with them:

        <i-scene experimental-webgl="true">

            <i-mesh id="first"
                has="box-geometry basic-material"
                size="2 2 2"
                rotation="30 30 30"
                color="0.5 0.5 0.5 1">

                <h1> Hello "mixed mode" </h1>
                <p>
                   This is DOM content, mixed with WebGL
                </p>

                <i-mesh id="second"
                    has="sphere-geometry basic-material"
                    size="1 1 1"
                    rotation="30 30 30"
                    color="0.5 0.5 0.5 1"
                    position="1.5 1.5 0" >
                </i-mesh>

            </i-mesh>
        </i-scene>

This will render a scene that mixes DOM with WebGL in the same 3D space, similar to my example fiddle here.

Changing the size attributes currently works. Internally it creates a new geometry with a new size, but as described above it is not performant if we want to animate geometry size without affecting children.

A unique feature of these custom elements is how sizing works. For example, we can write stuff like this:

                <i-mesh id="second"
                    has="sphere-geometry basic-material"
                    sizeMode="literal proportional literal"
                    size="2 0.75 3">
                </i-mesh>

where sizeMode specifies the size mode per axis, which causes size values for each axis to have different meaning. A “literal” size mode for a given axis means the value for the axis dictates the literal size of the object along that axis (without affecting children in the sub tree), and a “proportional” size means that the object’s size is proportional to its parent.

In this way, children in a sub tree are only affected if they have proportional sizing, otherwise not if they have literal sizing.

So in the previous example, the mesh has a literal X size of 2, a Y size of 75% (0.75) of its parent, and a literal Z size of 3.

Some of you might remember this sort of sizing from the now-abandoned open source projects Famous/famous and Famous/engine. I liked this sizing concept, so I adopted it, and will also be adding other size modes for doing some neat tricks.

TLDR, I would like for resizing of geometries to be performant and to not affect children in the subtree, and to be able to do this in a render loop. Later on I’d like to let the user optionally choose the type of internal size method applied (scaling vs making new vertices). In some cases, scale can work and boosts performance. In other cases, scale might have an unwanted effect on textures, so resizing the geometry (making new vertices) might have better results but less performance. It may also be possible to scale both geometry and texture…



How can it be implemented?

A couple ideas:

  1. Geometries can have a matrix that is multiplied by their owner Mesh’s matrix when calculating world transforms during a render (f.e. in a render loop), rather than a matrix that is used only before first-render. Basically a Geometry would be treated like an Object3D on it’s own, branched away from any of the Mesh’s children.
    • A downside is that shared geometries would all have the same scaling when the geometry belongs to multiple meshes anywhere in the scene, which may be unwanted and users would have to waste memory making new geometries so that meshes have their own geometry instances.
    • This would replace the current Geometry instance API with one that can be modified in the render loop to create animation.
  2. Meshes would contain an extra geometry-specific matrix. The matrix would be applied to Geometries as if they were branched Object3Ds.
    • This solves the downside of the previous option. Geometry transforms would be done using API on the Mesh instance, not on the Geometry instance.
    • The existing Geometry instance API would still exist, and would only be useful before the first render and not in a loop, as currently. Instead we would use the Mesh API to transform whichever geometry it contains.
    • This allows geometry sharing geometry transforms contained to their Meshes, which means better possible performance because one Gfeometry instance can be individually transformed on a per-Mesh basis.

Rotate and scale geometry
#3

There are ways around this without adding new features which is why you were asked to post this here instead of Github.

First of all, you should be using BufferGeometry rather than Geometry.

Next

Scale on a Geometry can currently only be set once and not in a loop, so we can’t get a performance boost using this feature.

The docs that you link to say:

Scale the geometry data. This is typically done as a one time operation, and not during a loop Use Object3D.scale for typical real-time mesh scaling.

It doesn’t say that you can’t do this more than once, or that you can’t do this in a loop. It’s just expensive so you shouldn’t.

Finally down to one possible solution - create an extra Group as the parent of the Mesh you want to scale independently.
Make the Mesh and the Group have the same initial position and rotation. Move any children of the Mesh to the Group instead. Now your mesh doesn’t have any children so you can scale it as you like, and if you need to move, rotate or scale everything then you do that to the Group instead.


[SOLVED] How do we swap the geometry and material of a mesh?
#4

Exactly. This is what I tried to explain on the a-frame meetup the other day.

Here’s how it looks like code-wise:

var group = new THREE.Group();
group.add( mesh );
group.add( children );
scene.add( group );

If you set mesh.scale.set( 2, 2, 2 ), the children won’t get affected. And if you want to move the whole thing, use the group group.position.set( 1, 1, 1 ).


#5

Thanks @looeee


#6

Oops, I was trying to do it the wrong way, in a way similar to the Object3D API:

geometry.scale.y = 2

which is why it didn’t work in the loop for me. It’d be nice for parts of the API to be consistent, following a pattern, so intuition works.



That can get unnecessarily complicated with bigger subtrees.

Suppose I have a tree of Meshes, 5 levels deep. Then, suppose I want to resize every mesh independently in the tree, but I want them to be hierarchical so that they rotate appropriately based on their parents’ rotation. The Group idea starts to become more complex, the dev has to do more work, and there’s more maintenance burden.

Imagine consuming a 3rd party component (a tree of Object3Ds) then having to modify the tree and setting a bunch of new transforms on all the new Groups: this is too much complexity.

What sort of authors consume 3rd-party trees? Library authors like those who make A-Frame and consume a DOM tree that should effectively mirror a Three.js tree. Having to hack Groups into 3rd party trees will be highly unwanted for these sort of developers.

Contrasting Three scenes to DOM CSS3D scenes, we can size Elements in a hierarchy of nested DOM Elements (using CSS preserve-3d and transform) without affecting child sizing. This doesn’t require transformation of a DOM tree into another tree just to achieve independent size changes. This affords us encapsulation of style versus structure, which is great! This is what makes HTML+CSS powerful most of the time (in some cases restructuring is unfortunately needed, but it isn’t common because that is what CSS aims to avoid).

See CSS Zen Garden for examples of CSS changes without restructuring the existing DOM. I’d like to achieve this separation of concerns without having to modify trees that are possibly given to me from other 3rd parties (I’m making a library), because modifying those trees introduces unwanted complexity.

The library I’m working on allows end users to write something like the following (similar to A-Frame in some ways):

<i-scene>
    <i-mesh size="20 20 20" rotation="..." position="...">
        <i-mesh size="30 30 40">
            <i-mesh size="30 30 40">
            </i-mesh>
        </i-mesh>
    </i-mesh>
</i-scene>

In this example, the end result will be rendered with WebGL (Three.js).

When we implement these elements, we have some options for implementing non-propagating sizing:

Option 1

Based on the above Grouping idea, we can tell users that they need to modify the tree if they want independent sizing. We tell them to write something like this:

<i-scene>
    <i-node rotation="..." position="...">

        <i-mesh size="20 20 20">
        </i-mesh>

        <i-mesh size="30 30 40">
            <i-mesh size="30 30 40">
            </i-mesh>
        </i-mesh>

    </i-node>
</i-scene>

Where <i-node> is roughly equivalent to Object3D, just a transformable node with no specific thing to render.

This option only works in some cases. We need to consider what happens when people make re-usable components for other people to use: When a developer imports a 3rd party Custom Element containing ShadowDOM for its inner tree, or a React component with inner “HTML” as JSX and props.children, or an Angular directive, or a Vue component with slots, the users of these components are not able to modify the internal structure of the given 3rd-party component without introducing seriously brittle hacks that component libraries aim to avoid in the first place. Component internals are essentially private.

This makes it nearly impossible for users of 3rd-party components to be able to independently size Meshes in a tree when the tree is composed of components.

Option 2

As a library author, we can allow the end developer to independently size items in the tree without requiring them to modify the tree structure, so they only ever need to write

<i-scene>
    <i-mesh size="20 20 20" rotation="..." position="...">
        <i-mesh size="30 30 40">
            <i-mesh size="30 30 40">
            </i-mesh>
        </i-mesh>
    </i-mesh>
</i-scene>

If we implement this with Three.js and using the Group strategy, it will introduce high complexity to the library implementation so that the library can map from a DOM tree to a more convoluted Three tree. This would involve having to forfeit direct mapping of DOM mutations to simple Three tree mutations; we’d have to forfeit directly mapping Node#appendChild calls to Object3D#add calls, forfeit pure reliance on Object3D#children and keep track of our Three children in a special way, etc. It’s not ideal, but it would make usage of the library easy for end users.

Option 3

We can fix this in Three.js, so that library devs and app devs can both benefit from a lower-level abstraction that would be simpler than both option 1 and 2. We can scale Geometries in a way similar to how we do currently using Object3D, rather than re-calculating all vertices.



transforming geometry cheaply

Being able to transform Geometries directly, as cheaply as Object3Ds, would be great. After looking at the code, I see how currently applying a transformation to every vertex of a Geometry is heavy.

What if we treat mesh.geometry like a branch in the tree, and treat the Geometry like an Object3D with a world matrix, then allow people to transform the Geometry just like they do other objects? F.e.:

mesh.geometry.scale.y = 2 // similar to Object3D and cheaper than current!

#7

I don’t think it’s as complex as you are making it out to be. You can create a function something like:

makeObject3DIndependantOfChildren( object3D ) { 

    if( object3D.children.length > 0 ) {

         const group = new THREE.Group();
         group.add( object3D );
         group.add( object3D.children );
        
         object3D.parent.add( group ); 
    }
    
    // return a reference to the group that you can use in case you still
    // need to transform the object with it's children
    return group;

}

Then you can transform the mesh as you please without affecting the children.

You could proceed to create utility functions like:

scaleObject3DIndependantly( object3D, newScale ) { 

    makeObject3DIndependantOfChildren( object3D );
    object3D.scale.set( newScale );

}

This idea would need more work to be robust as part of a framework of course. But the basic idea should work.


#8

I know it can be done, but it’s not ideal for maintenance and makes the tree harder to reason about when it isn’t in it’s original form. Just imagine if you inspected-element on an HTML app, and the tree structure what not what you wrote? It would be strange (people complain about this with <table> elements, and it is a major reason why Custom Elements do not work with tables.)

If I want to make a CSS-like tool that depends on tree structure, there will be more complication there, more room for bugs. If the third party goes to grab some object in some expected place of a tree, it may not be there anymore, causing unexpected bugs. Etc.

I know how easy it is to do what you suggest, and in fact your solution may be fine for applications made by one person, but as soon as we have developer relying on other developers’ trees, this will increase opportunity for error, and decrease developer productivity because they will have to examine what other codes modify the tree in order to understand what form the tree will finally have in order to make a further complicating change. This will simply make applications more brittle the larger the scene graph tree gets (and the more developers depend on each other’s trees (f.e. think React, Angular, Vue, Preact, etc).

This would greatly complicates how component development is done. F.e. Authors making React components, Angular components, Vue components, etc, will all need to implement code to map from their component tree to non-matching Three.js trees, and that will be a huge maintenance pain for people trying to add new features to existing trees. For example, It means developers will have to make additional properties besides parent and children on Object3D, f.e. realParent and realChildren in order to map to the original tree, or other contrived things that will lead to spaghetti.

Mapping one-to-one from parent/children of an Three.js Object3D tree to parentElement/children of an Element DOM tree is simply much more desirable.

I know I can work around the issue by doing what you suggest, but it’s going to increase the surface area for bugs with every feature added, where each new feature relies on the user-supplied tree structure and mapping those features to an internal Three tree structure that may not be the same tree structure the moment some other developer makes more structure modifications.

IMO, It’s just not the right way to do this in the long term. Sure, I can get by in a small application prototype using your technique, but I’m working on something that I hope will be more future proof.



So, how can we do tree-structure-independent Geometry sizing as easily as Object3D scaling?


#9

All these Vue/React/Angular/etc developers already handle different threes for their components.
I do not see any issues on this side as you can always base your selectors on classes, but not on elements hierarchy.

The main issue I see is the application speed (we are talking about 3D graphics that always was quite slow). If I understand correctly the treejs stores the data internally in the way that allow it to be as fast as possible when the data should be send to GPU.

You suggests to modify geometries. The main questions I have:

  • Can be done as fast as before runtime (60 times a second)?
  • How hard it will be for treejs to re-create all internal buffer geometries runtime? (I expect lags runtime when the geometries are re-building)

#10

The way I’m imagining it is the only overhead would be similar to adding an extra Object3D in a scene, so the overhead of something like an extra updateMatrix call.

They wouldn’t be recreated, only their matrixWorlds would be updated, like with Object3D.

To make Geometry be like Object3D and have a matrixWorld, and the renderer would traverse mesh.geometry while rendering a tree. This so far is the simplest way I can think of in order to prevent everyone who builds libraries on top of Three.js from having to restructure trees just to resize Geometries.

I would really like to be able to scale a geometry without restructuring the scene graph and without recalculating all vertices.


#11

I’ve made something similar before. I needed a scale/size to size a rounded box on the GPU without stretching.

Basically i just extended the base shader i needed (like lambert) inserted the uniform and the line i’m sizing it. Then created the mesh class for such objects by extending THREE.Mesh, and adding a size Vector3 besides position, rotation and scale. The prototype implements the onBeforeRender callback, grabs the material there and applies the size to the uniform.

Edit: forgot to mention, you need to insert this resizing shader code in the vertex shader of distanceRGBA and depth in THREE.ShaderLib as well for shadows to work. But i think adding it to the begin_vertex snippet instead would be less redundant :grin:

The code you insert needs to be optional with #ifdef around, the materials being capable of it having something like USE_SIZE in the defines property then.


#12

(I know this is an old thread, but I’m new to the forum and it just surfaced for me, so mods can feel free to delete this if they deem it lacking usefulness.)

It sounds to me like the real solution is to enforce 1::1 Geometry::Mesh relationships, and to separate the concepts of Mesh and “tree node”.

At the end of the day, what matters to the renderer is the associated geometry, material, and world transformation (matrixWorld). The glue that holds them all together is the Mesh object. So regardless of whether mesh1 is a parent of mesh2 or not, it’s the final computed world transformations that have the last say. (If mesh.autoupdateMatrix is set to true, then this computation occurs every render.)

My point being that you can keep a “flat” scene filled with only Mesh objects, with autoUpdateMatrix set to false. Then you control the hierarchy through whatever external means makes sense for your structure. You could even use Object3D to make computing the matrices a breeze.

Then, when you go to render, process your tree matrices down through your hierarchy so that you arrive at a final world matrix representing the transformation you will apply to the associated Mesh. Take a peek inside WebGLRenderer and you’ll see this is exactly what happens with autoUpdateMatrix as true, only this way, you’re controlling the transformations the way you see fit.


#13

“7 months later”

This is something i dislike about discourse, when somone just edits something it will bump to the top again, didn’t knew this was a half year old.


#14

That’s effectively exactly what I need and what I was starting to imagine and describe here. It’ll be great to have this. No chance your code is available somewhere?

That’s actually turns out to be what ended up doing. Well, similar: I’m still attaching the Three objects in the same hierarchy as my external tree with Object3D.add(), but I’ve set autoUpdateMatrix set to false on all of them and am supplying them all with custom matrixWorlds from my external tree.

I was contemplating next to not .add() them to each other and keep them flat as children of the scene.

The problem is, I still have to implement the shader handling of an extra size variable like what @Fyrestar mentioned, so that the size doesn’t affect child objects. This will help me get better at working with Three shaders. :smiley: