Noob question about functional reactive programming, state (vuex), and threejs

[benwinding]

State-managed THREE.js is a goal of mine too, but I’ve found this could result in extremely poor performance and a huge amount of translation between the state and the THREE instance, here’s what I’ve tried…

Solution 1 (naively rebuild all objects)
Completely rebuild (not redraw) the scene every time the state changes.

• This is amazingly slow and inefficient
• Easy to implement
• Basically not an option for any slightly complex scene.

Solution 2 (the JQuery way)
A slightly better way is to use state transitions, where the specific change in state has a corresponding mutation function; eg:

TreeCountState += 1 --> Triggers --> ChangeTreesCount().

• This is relatively efficient
• Incremental to implement
• Unpredictable performance, as it relies on custom modifications to the scene

Solution 3 (Virtual DOM style)
Similar to this response, we could find a way to create something similar to a VirtualDOM, similar to react/vue/angular.

• Easy dynamic scenes based on JSON state.
• Nice separation of concerns, THREE js only would depend on a state object to render a scene
• This would require a lot of work to get the system to manage and track all the various assets in the scene, and change them when the state changes.
• Would perform ok probably, but latency would definitely be an issue.

Right now, I’m attempting to use Solution 2 and it’s not as hard as I thought, I have no doubt that solution 3 will be developed eventually, but depending on you performance requirements it might not be suitable for every application.

Can anyone think of any other methods?

[pailhead]

Is your approach 1 basically just straight up three.js?

[benwinding]

Well, not from the code I’ve seen.

Using the naive Solution 1; every tiny change would require everything in the scene to be remove and to be rebuilt from scratch using the new state. I haven’t seen much code do this three.js, have you?

After testing this approach, I’ve found the framerate drops very quickly, compared to simply removing the single change that occurred in the scene.

[pailhead]

I’m not sure if I understand the approach. The entire frame is rendered from scratch with threejs. The data structure though remains in place (scene graph). I think this is the opposite problem of what react solves (partial updates)?).

[benwinding]

Thanks for your responses @pailhead, okay I think I understand now, yes, three.js redraws the entire scene from the scene graph to the canvas on every frame.

The first approach (Solution 1) I was talking about is simply reconstructing the entire scene graph itself from scratch every time there’s a state-change. This makes it much easier to manage as any state will be correctly reflected in the canvas. However there’s a massive overhead in completely reinitializing the scene on any tiny change. You’re right, that approach is not like react/vue/angular.

Eg; I have 1000 trees and then the state changes to 999 trees, the scene graph needs to be destroyed and rebuilt from scratch, which is crazy inefficient.

It would be much nicer to somehow link the application’s state directly to three.js’ internal scene graph, is that possible? (which is the idea behind Solution 3)

[Also edited first response to be a bit clearer]

[looeee]

@benwinding

I have 1000 trees and then the state changes to 999 trees, the scene graph needs to be destroyed and rebuilt from scratch, which is crazy inefficient.

You can tree.visible = false rather than removing and adding things to the scene, it’s much more performant.

[underscorefunk]

Recreating all of the geometry every time is a performance nightmare. I think that trying to create a VUE or React interface for the scene graph is more coupling and complexity than is required. A solution that I came up with that works relatively well is to use a least power approach. Have a state object with properties and have those properties yield geometry. Then, have mutations to the state, trigger mutations to the scene graph. It’s not a “perfect” solution, but watching properties or using dispatching actions is relatively trivial to do and has a linear flow of data. And, you can process the scene in a functional style. It makes it all pretty doable.

[pailhead]

@benwinding

The first approach (Solution 1) I was talking about is simply reconstructing the entire scene graph itself from scratch every time there’s a state-change.

I thought it might be this but i wasn’t sure. I had a colleague who suggested we do something similar with meshes. Copy and pass the entire blob with all the vertices and such upon a state change with redux.

I’m not ridiculing this (even though we agreed that its ridiculous ) but i did find it interesting that that idea even came up.

The reactive / functional / declarative paradigm is humming along, and does wonders for everyone who’s rendering spread sheets. Three.js is the very antithesis of all this, and i’ve only seen nightmares when it comes to marrying the two.

How webgl works is (i think) akin to timestamps in redux. You do your actions to load the buffers, but afterwards, you’re not moving and removing stuff from the gpu, it sits there, you have a pointer, and you fetch it from time to time. The analogy would be something like:

interface INodeState {
  id: string
  teddyBear: Toy
  reference: THREE.Object3D
  timestamps: {
    position: number,
    name: number,
    rotation: number,
    ...
  }
}

The idea is something changes, you compare the timestamps, and fetch the value from the (mutable) three.js reference.

@benwinding

However there’s a massive overhead in completely reinitializing the scene on any tiny change.

However you could improve the overhead here and keep the idea somewhat. I imagine you were deleting your objects and letting them be garbage collected. completely reinitializing the scene ← i imagine you were doing this with new THREE.Mesh(...).

You could drastically improve the performance if you recycled these objects. You can have a pool of three.js objects, instead of deleting them you can release them to the pool. Instead of making new you can acquire them from the pool (and make new only if there arent enough free nodes in the pool).

[benwinding]

Thanks again for the detailed responses guys!

@looeee

tree.visible = false

This is a much better idea than add/removing items

@pailhead

How webgl works is (i think) akin to timestamps in redux. You do your actions to load the buffers, but afterwards, you’re not moving and removing stuff from the gpu, it sits there, you have a pointer, and you fetch it from time to time.

So you could create your own representation of the scene graph and periodically check for changes using Timestamps? Sounds like an interesting idea

@pailhead

You could drastically improve the performance if you recycled these objects. You can have a pool of three.js objects, instead of deleting them you can release them to the pool.

This sounds like the way I will go for some parts of the scene to improve performance, similar to managing thread pools I suppose.

@pailhead

The reactive / functional / declarative paradigm is humming along, and does wonders for everyone who’s rendering spread sheets. Three.js is the very antithesis of all this, and i’ve only seen nightmares when it comes to marrying the two.

I’m starting to see the nightmare…

But I still have hope that a more reactive approach could one day be used like the shift from procedural JQuery to declarative components (react/vue). What are your thoughts on this project (react-three-fiber)? It seems to have a declarative API, although I’m not sure if everything is being regenerated on state changes.

[pailhead]

@benwinding

So you could create your own representation of the scene graph and periodically check for changes using Timestamps? Sounds like an interesting idea

I’m thinking actually of just a flat list of all the Object3Ds that are in the graph, doesn’t have to be necessarily connected as a graph, (you could access that by querying the reference).

@benwinding

What are your thoughts on this project (react-three-fiber)?

Wow, i wasn’t aware of this. I like the description and the claim, my first thought is always that this would be slower than three.js. The first thing i notice is that you pass some camera props into the canvas, which is how i was approaching this problem - there needs to be a react component that acts as a view into GL. Here (at first glance) seems like it’s coupled with the canvas which i don’t like. I’m trying to make something like <ThreeView> and <ThreeCanvas> where several views would have their own cameras but all render to the same canvas.

[underscorefunk]

ThreeJS does take a scene and camera and push it into a renderer. You can, in a functional/declarative way, modify that scene as a pure function. Sooooo, you can totally just set up a queue of actions, have them pipe through the scene, and then push it to the camera to render the frame.

It works really well.

Also, you can assign objects with unique IDs and then track them in state.

You kinda get the best of both worlds.

[looeee]

@pailhead

The reactive / functional / declarative paradigm is humming along, and does wonders for everyone who’s rendering spread sheets. Three.js is the very antithesis of all this, and i’ve only seen nightmares when it comes to marrying the two.

I thought the same until I saw react-three-fibre. Especially this example:

space game - CodeSandbox

Then again, I haven’t explored the code that deeply, and I’m not familiar enough with declarative coding or React to judge whether this is an improvement over the same app built in pure JS.

If I had a spare day or two I’d love to spend some time rebuilding that in three.js so that I could make an informed comparison.

[underscorefunk]

Wow. That’s an impressive demo.

[benwinding]

Thanks for all your help @pailhead, @underscorefunk etc.

Here’s the dumb little site I was working on, let me know what you think:

Demo: https://benwinding.github.io/wealth-visualizer/
Source: https://github.com/benwinding/wealth-visualizer

Code notes:

• Using Vue.js
• Not very declarative, but does fully separate state and the three.js logic nicely.
• Input fields dynamically update the state in the scene instantly (visible true|false is much easier)

react-three-fiber looks like it would be much more declarative, but could also have big performance hits too. Anyway has been a fun little project I always wanted to do.

[TheHappyKoala]

Super cool! Would be even cooler if you could change the currency, though… You could even have a stack of dollars, and then next to it a stack of [insert massively weak currency of your choice], so that you could get an idea of the relative values of different currencies. Maybe you could include historical currencies, too… Like the Deutch mark of the Weimar Republic… Anyhow, I’m ranting, but cool project!

[underscorefunk]

I wish I knew more about Vue so that I could see the connection between state and modification of the ThreeJs scene graph. Can you summarize how you’re pushing state changes to vue graph changes?

Btw, the project is really cool. Nicely done!

[Usnul]

Regarding declarative style and re-building the whole scene every frame. My engine does exactly that, and it does that for exactly the reason of performance. The key is to prevent THREE.js from doing any unnecessary work when an object is added to the scene, and to re-use objects whenever possible .

When it comes to state and mutators - I think there’s nothing complex there, it might take a while to create all necessary mutators, but not because of their individual complexity. It will depend on your usecase how many mutators you will need.

[benwinding]

Thanks guys! Yeah it’s taken longer than I thought but it’s a fun little toy to show people.

@TheHappyKoala, they’re some great ideas, I should do Australian notes, since I am Aussie lol

@underscorefunk, sure, the code basically works like this:

1. meshes.init() initializes all the meshes in the app are generated, added to an array, all hidden and added to groups and to the scene.
2. The UI component has a property being watched, when it changes; meshes.setState(newValue) is called.
3. meshes.setState(newValue) then sets visibility of all tracked meshes according to the new state.

Seems to work pretty well I guess

[underscorefunk]

Very neat. Thanks for the breakdown. That seems to be a common pattern of creating all of the assets, adding them to the scene, and then mutating them on state changes. I’d say it works well!

[drcmda]

@pailhead

The first thing i notice is that you pass some camera props into the canvas, which is how i was approaching this problem - there needs to be a react component that acts as a view into GL. Here (at first glance) seems like it’s coupled with the canvas which i don’t like. I’m trying to make something like <ThreeView> and <ThreeCanvas> where several views would have their own cameras but all render to the same canvas.

These are just defaults. You can bring your own camera or practically just render into a THREE.Scene, just like you do with ReactDom.render(…, domNode). The defaults are there to get going quickly but it’s absolutely modular. I think it solves pretty much all issues discussed here, each component can easily be bound to state. And with React that covers many other areas as well like suspense, loading, async, etc.

ps, here’s an example of what you wanted: React-Three-Fiber Gestures - CodeSandbox

Components render over one another, some of them having their own cameras.

or this one: Viewcube - CodeSandbox renders a ViewCube on top of the scene by portal into a plain THREE.Scene.

Basically, there are no regressions. If it works in THREE, it will work in r3f.