Noob question about functional reactive programming, state (vuex), and threejs

I preface all of this by saying that I’m new to this community and got completely sucked into the JS development world a month or so ago and three.js recently.

I’m looking to build a map editor of sorts and I’ve been looking up different design patterns so that I can setup my app so that it’s easy to reason about and extend. I feel like I’m getting a bit of analysis by paralysis from searching the web to try to find answers about state driven scenes and approaches to bind data to the threejs output. :frowning:

My question is:
I would like to have my app state (basic position, rotation, material name, cursor position, etc) in a single source of truth (vuex state management) so that I can easily dispatch mutations and let the scene graph and render be a function of the state. This unidirectional data flow makes everything super clear, easy to follow, and awesome (or so I think in my naivety).

I’m trying to figure out what the most economical way is to connect the state to the scene graph.

In the simplest and most pure way I’d just trash everything and re-add it as a total pure function but that feels like it would be exceptionally “unperformant”

What I’d love to do is have something be fairly declarative and let threejs mutate its own components are required by retaining components in the scene graph but syncing them with the state.

In my mind I’m thinking:

  • Do I track objects by uuid to see if they exist and create them if they don’t
  • Do I write wrappers that match state to uuid on updates checking for diffs or just resetting all of their values based on state? Is that traversal really inefficient or maybe I create a lookup table to sync state → graph object values?

One thing I’d like to do is use Vuex as my statement management so that I can use Vue components to mutate some of the scene objects easily. Since Vuex is a mutable state compared to react, I thought it might work more easily with the mutable scene graph (though I understand that some components of objects like their position/Vector3 are immutable objects).

I feel like I’m missing something obvious and spinning my wheels as a result. :frowning:

Am I being silly for wanting to do this?

Thanks to anyone who has any insight on this and for considering my question.

I have never even heard about Vuex before, but I find your post interesting nonetheless.

Is your vision to build something that wraps all functionality of three.js into the Vuex pattern, or do you only need an interface to a given scene? Or is it something in between?

To the first interpretation: I think trying to wrap all of three.js into a very specific architectural pattern is almost guaranteed to fail. All restrictions imposed by three.js, as well as all optimizations and other stuff three.js does under-the-hood, will make it harder to do something like that. In that case, if you were really serious about doing web 3D graphics in a fundamentally different way, you would probably be better off working directly with WebGL.

To the second interpretation, which is basically using three.js for rendering a state: Most applications interact with three.js through event listeners and requestAnimationFrame. So I suppose you could “listen” to state changes, handle them by applying to the three.js objects and finally either render directly or requestAnimationFrame. (I may be totally misunderstanding the question, though.)

No, it’s not silly. I have a THREE app where the gui is written in React, and I use Redux to sync the scene with the UI and thereby make it interactive, and I don’t mutate state… So while I’m no VUE ace, I don’t see why you shouldn’t be able to accomplish what you have in mind as you don’t have to worry about immutability.

What I do is that the scene is in its own module that has no knowledge of React. The module takes a canvas element as an argument and does not care about where it came from, as long as it gets one. Then I import the Redux store, read from it before the render loop starts, and then at the end of the render loop is dispatch whatever has to be dispatched. Beauty of this approach is that if I wanted to switch to Angular or whatever, I could do that without issues, as the gui and scene parts are separate parts of the applications that are glued together by the store.

Then I have a renderer component that passes the canvas element to the scene module:

import React, { memo, ReactElement, useEffect, Fragment, useRef } from 'react';
import scene from '../../scene';
import './Renderer.less';

interface RendererProps {
  scenarioName: string;
}

export default memo(({ scenarioName }: RendererProps): ReactElement => {
  const graphics2DCanvas = useRef(null);
  const webGlCanvas = useRef(null);
  const audio = useRef(null);

  useEffect(() =>
    scene
      .reset()
      .init(webGlCanvas.current, graphics2DCanvas.current, audio.current)
  );

  return (
    <Fragment>
      <canvas ref={webGlCanvas} />
      <canvas ref={graphics2DCanvas} className="graphics-2d-canvas" />
      <audio ref={audio} style={{ display: 'none' }} />
    </Fragment>
  );
}, (prevProps, nextProps) => prevProps.scenarioName === nextProps.scenarioName);

It’s more that I’m only going to be touching specific parts of threejs with vue. Vue isn’t going to totally manage or wrap threejs. I think that’d be a nightmare. Instead I basically want vue/vuex to manage state for a specific property that I would then map to a specific part of threejs. Namely, things like the position of things, the name of a material (which would be defined externally from state, etc). Your second interpretation is bang on.

The thing that I’m trying to figure out is, how exactly does one best listen to state to do that update during every tick of the animation loop.

If I have store.state.objectA with a position property, but the loop has just a ‘render scene’ that accepts state… how does one link up store.state.objectA.position to scene.children.objectA to update its position in a sensible and performant want. Mostly because I don’t think there’s a way to know if one specific part changed (and I don’t want to need to check if a specific parameter changed with a diff or anything like that).

That’s sort of what I’ve got going locally. But I’m a little bit stuck at, when the scene has a new state, how does (or should) the scene get updated from the new state. Am I better off to start with a fresh scene every render pass and then add the objects that are in it? Or am I better of to carry the new scene forward as a recursive loop, and then mutate the values. And if I’m mutating the values, is there a “best” way to do that?

Apologies if I’m using the wrong terminology here and being a bit dense. :confused: A lot of what I’ve read so far is, “Do this and then this… [magic happens], so now it works,” and I’m stuck on trying to add the magic. :wink:

… I feel like if I can get this pattern to work it’ll result in a display component totally separate from state and life will be ‘gooooood’. ;D

Creating a fresh scene with every render pass would be very unperformant; my app is a simulator where you can choose from an array of different scenarios, and I only reset the scene when the name of the scenario has changed, as you can see in the renderer component I included above.

Here’s the scene module; it has an api that react interacts with that is simply made up of an init and reset method… To get to the state related goodies that I suppose you’re interested in, just search for store and you will see how at the top of the render loop get the ui state which I then use to update rotations, physics and all the rest, and then at the end of the loop I do a few store dispatches to sync the simulation with the UI.

I apologise in advance for the beehive that is the code in the scene module, but hopefully you can make some sense out of it, and if you have any questions, shoot.

Forgot to include the link to the scene module: Harmony-of-the-Spheres/index.js at 65494f32d9142f1b40ae710e335e1423f9f39a25 · TheHappyKoala/Harmony-of-the-Spheres · GitHub

Now who’s looking dense :rofl:

If you know Vuex well and three.js less well, and the state changes come from Vuex, I suggest you let Vuex be in charge, rather than trying to make sense of the Vuex state from the render loop. Doesn’t Vuex allow you to somehow listen to single minor changes, so that you can update the scene graph and materials on the fly, independently of the render loop?

I’m new to all of it so maybe that’s something that I might need to look for. Maybe I need to find a way to register those properties as being ‘reactive.’ so that they auto update.

I think I’m half stuck between purely functional stuff/redux, reative stuff like RxJS, and mutable stores like Vuex.

Your insight and perspective here is super helpful. Thanks so much for taking the time! I’ll post my findings when I do some experiments this week.

I wish you luck! I have absolutely no experience with web development platforms, so my “insight” is only about HTML/CSS/JS/three.js (plus some WebGL) and programming in general. But I assume your expression of gratitude was directed at @TheHappyKoala too.

I’m appreciative of both of your insights. I’m still feelin pretty confused but I’ve got more direction to go on now. :slight_smile: I’ll document my findings and post updates as things develop.

1 Like

I worked through the problem a bit and I think I came up with something that might work. There were a few gotchyas that I found. I had to resign to the fact that there would be no way to directly and reactively push state to threejs. So, instead I opted for a series of queues that add synchronizers to make threejs update its objects.

Here’s a sample of the idea. Would love to get your thoughts on it. Again, I’m aiming to go with the least code, the most flexible, and to keep things really really simple.

https://pastebin.com/T6VSkZma

1 Like

Like the concept. Well, what’s not to like about creating a layer of abstraction that is more semantic and easy to work with… Have you gotten to developing a working prototype?

Right? :slight_smile: I think something like this is necessary if you want to be able to share state across the network so that more than one person can see the same output. I’m hashing out a working prototype right now. :slight_smile:

1 Like

Yaaayyyy… Chances are I could use your solution to tidy up the scene module, so I’m cheering for you.

1 Like

Initially I was thinking… Actions -> Vuex state changes -> mutations trigger/enqueue THREE scene changes.

I’m now seeing that trying to predict the future and using vuex for reactivity is needless preemptive complexity.

Currently my direction is… setup a system with ‘commands’. When a command is issued a state mutation happens and a THREE scene graph mutation takes place.

This puts all of the impure parts in the command system and commands can be passed around. Instead of needing vuex and firebase to create synchronizations I can use websockets and just pass the commands around. It means adding UI components is going to be a bit of a pain… but simplicity and a bit of work instead of complexity/libraries, and other magic might be the way to go.

1 Like

I did a test so far of setting the camera position according to the cursor and it totally worked. :smiley:

3 Likes

I’m super glad to see people exploring this problem publicly! Bulk of my work for a couple of years was marrying redux and threejs :frowning:

2 Likes

Super happy to help document the discovery process. If you have any insight to add from your experience, please feel free to post it here… even if it’s a bit of a tangent.

I was surprised that there don’t seem to be many straight forward design pattern examples for using state models to manage three.js. :confused:

Sadly, I’m pretty new to JS and very new to Three.js. Hopefully my meandering and hacking through this stuff won’t turn into a total mess. LOL :smiley:

1 Like