Creating moving parts in blender that will move in response to user input in 3JS

drcmda · June 6, 2020, 9:51am

If that doesn’t work, it sounds like I should use react or a similar program to get the job done. I am surprised that Blender does not include the capability. But I guess that is not a primary function of the program.

blender can most certainly run animations, but a timeline animation is often the wrong tool for interaction. nor is it easier to make and control - it can be harder. but it depends on your actual use-case. im just saying, if i had a lever that the user is supposed to pull up and down, that’s nothing more than a rotation node in blender and a gesture in javascript.

phil_crowther · June 7, 2020, 9:00pm

A rotation node is probably exactly what I am looking for. Do you know how to access or manipulate those nodes in 3JS?

wassfila · June 8, 2020, 8:37am

Hi,
I faced exactly the same issue a while ago, and this is likely to be exactly what you’re looking for.
My goal was to design once in Blender to avoid any tweaking in text files of js. Then connect a parameter to any sort of input, in the demo it’s a gui.dat but my code simply listen to events.
I had to fallback on Custom Properties due to a limitation but that was a while ago (5 Months ago).

I tried to make these params generic, not only rotation or movement but also colors and light,…
here is the full github repo with examples and its readme with gif demos and links to live demos

as a hint, think of having your animation linear in Blender.

the usage is event based :

	let controller_pull = gui.add(config, 'pull',0,1);
	controller_pull.onChange(value => {send_custom_event("three_param",{name:"Axis", param:"pull",val:value})});

drcmda · June 8, 2020, 11:28am

you just give it a name and it will be a generic object3d or a group in the gltf export. alter the rotation property and that’s that.

wassfila · June 8, 2020, 12:15pm

That is correct. But doing it that way you’ll have to define for example the rotation limits and axis in text mode or hard-coded in js. The advantage of exporting an animation is to allow the 3D designer to define the extent of the motion and the developer to connect it to any sort of user input.
Which method is easier depend on the scalability of the project. For a simple example I’d go with your suggested method.

phil_crowther · June 8, 2020, 4:41pm

The aircraft that I save in glb/gltf is composed of several parts. Does 3JS load a list of part names and references when it loads the file? Or do I have to parse the file to do that?

It sounds like you are saying that, once the file is loaded, I can access the parts separately. What command do I use to do that? Something like Object.part.rotation.x = value? (I tried that specific command, but it didn’t work.)

Thanks!

phil_crowther · June 8, 2020, 4:51pm

This approach looks like it could be pretty useful.

Currently, I am saving animated parts in separate files and using routines to position them on the aircraft. However, for some reason, with this particular part (the elevator), I am having trouble rotating it correctly along all 3 axes. This is odd because I have been able to attach and rotate a transparent propeller disc to the front. It rotates only on the Z-axis (bank). The elevator rotates only on the X-axis (pitch). But it is vertical of the centerline - so that may be contributing to the problem.

In any case, the animation could help me avoid the mathematical challenges of correctly positioning and rotating the part within the scene. I will give it a try.

wassfila · June 8, 2020, 5:53pm

That’s exactly what I design this for, so that the position and motion (in my case light bulbs), is done in a professional CAD tool like blender and not wasting my time in adjustments of things I need a launch loop to see or develop an edit tool to modify.

Also, another hint regarding your comment on separating different parts. You know best what is more practical to your use case, but I do not see the need of separating the parts, that might get more complicated. GLTF is a standard made to combine a whole scene and should handle all their hierarchical positioning. I know in 3D libraries it’s not always easy, but GLTF in three.js and Blender do have exactly the same hierarchy.
An example screenshot below from a scene in blender with Collection and the reversed triangle is a Frame.
scene_in_blender

And once in Three.js you have two powerful method to get you objects back
scene.getObjectByName() if you already hardcoded or configured your object names, or even nicer, an exploration loop where you could discover all sort of items in your scene and manage them accordingly, note in the example the usage of the userData that provides you back what you configured as Custom Properties in Blender :

github.com

HomeSmartMesh/smart_home_3d_webapp/blob/f34d24d913edbdc6914e4086f63c82c973db5376/src/three_app.js#L468


	}

	requestAnimationFrame( animate );

}

function sendMeshLists(){
	var mouse_events = [];
	var mqtt_topics_map = {};
	var hue_light_map = {};
	scene.traverse(obj => {
		if((obj.type == "Mesh")&&(obj.userData.mouseEvent == 'true')){
			mouse_events.push(obj);
			//console.log(`three_app> mesh '${obj.name}' with mouseEvent`);
		}
		if((obj.type == "Group")&&(obj.userData.mouseEvent == 'true')){
			mouse_events.push(obj);
			//console.log(`three_app> mesh '${obj.name}' with mouseEvent`);
		}
		if(typeof(obj.userData.mqtt) != "undefined"){
			mqtt_topics_map[obj.userData.mqtt] = obj.name;

I you have any issue or question, let me know I’ll help you.

phil_crowther · June 9, 2020, 4:13pm

Okay, I may be doing this all wrong, but here is what I have done:

I created a keyframe animation in Blender named: tailHE_anim
This rotates the elevator on the X-axis from -24deg to +24deg.
It is a linear progression as you described with a range of 0 to 60.

In my program, when I loaded the file with the animation, I included the following steps:
mixer = new THREE.AnimationMixer(ACObj0);
clip1 = gltf.animations[0];
action1 = mixer.clipAction(clip1);

I assume that action1 is the address to the animation?

What do I do now?

DolphinIQ · June 9, 2020, 5:21pm

Then you can activate your animation whenever you want with action1.play() or stop it with action1.stop(). Have a look at all available control methods in AnimationAction docs, and especially at .timeScale.

Lastly, to make sure that three.js updates your animation every frame, use
mixer.update( deltaSeconds )
in your animation loop.

wassfila · June 9, 2020, 5:32pm

in your case, I assume that you do not want the plane tails to be moving by themselves with time in that case you should not be calling .play() and .stop(). Instead, you’d like to move the tail to a given position that is decided by the user given a certain input. That input, let’s say in my example is between 0 and 1, will be mapped to the animation duration.
For that you could call mixer.setTime()
The example below is the code from the running example, I showed on the gif animation

github.com

HomeSmartMesh/web_three_interface/blob/b888c8da1c7abd8e64f6b3ff8a2a612319c79aa0/09_param_rot_scale_view/src/three_app.js#L422


	if(anim_params[obj_name].type == "mixer"){
		const mixer = anim_params[obj_name][param_name].mixer;
		const duration = anim_params[obj_name][param_name].duration;
		let time = val * duration;
		if(time<0){
			time = 0;
		}
		else if(time >=duration){
			time = duration -  0.00001;//This is a bug in THREE animation as duration value turns back animation to 0 and not clear how to set it at the end of the animation
		}
		mixer.setTime(time);
	}
	else{
		const obj = anim_params[obj_name].object;
		set_obj_view(obj,val);
	}
}

export{
		init,
		animate,

An additional hint, I planned the input value to be between 0 and 1. For some reason setting the value 1 if mapped to the full duration would restart the animation from the beginning, therefore the patch on line 420 to lock the position at the end of the animation.

phil_crowther · June 10, 2020, 3:30am

Exactly! That’s what I want. So calling mixer.setTime() should be all I have to add to my code?

I couldn’t get the animation to work on that program (who knows why?), but was able to get it to working on another. I tried adding:
mixer.setTime(.5);
but that didn’t seem to have an effect on the file. It sounds like it should have set the animation at midpoint?

EDIT: I got the animation to working on the target program (I needed to add the clock information). So if I include the command action1.play(), the animation will play.
And I realized that the mid-point value for mixer.setTime(t) should be 1.25 seconds (1/2 of 60 frames / 24 fps). But I was still unable to move the elevator to a specific location.

wassfila · June 10, 2020, 6:34am

It might be due to missing initializations. Also the time should be mapped over the animation durarion. I don’t see what you are referring to by “that” program. What I can do is review your code, either the original one, or a minimal sample on which you reproduce the issue.
I’m sure you’re pretty close, and with this technique you can scale up to dozens of parameters without touching any js code.

phil_crowther · June 10, 2020, 12:34pm

The program I am working on is this one:
Flight Simulation Demo
The “other” program is this one:
Flight Display Demo
Neither version has any Blender animations yet. My animations for the props and engine use parts created separately in the 3JS program…

Yes, I believe that my problem is with the initialization. Eventually, I will want to be able to reference several different animations, elevator (pitch), rudder (yaw), ailerons (bank) and flaps. From what I understand of Blender and 3JS, it is possible to have several different animations and to choose between them. And mixer.setTime() appears to be exactly what I want to access each one of them.

I think the missing piece is that I have not figured out how to access/address the specific animations that I will access with mixer.setTime(). For now, my “setup” only invoIves the command:
mixer = new THREE.AnimationMixer(aircraft_object);

I can run the animation by adding the following:
var clip1 = gltf.animations[0];
var action1 = mixer.clipAction(clip1);
action1.play();

But that does not appear to access a specific animation. My understanding is that the mixer is supposed to access all animations. So if I had several animations, this would just run them all. It appears that I need a setup which allows me to separately access each Blender animation that is included within the gltf/glb file.

If it helps I can put together a simple program that merely displays the aircraft in a blank space.

wassfila · June 10, 2020, 4:55pm

Ok, I see you’re getting there, step by step.

The secret to re-associate animations to objects in three js is in this line :

github.com

HomeSmartMesh/web_three_interface/blob/b888c8da1c7abd8e64f6b3ff8a2a612319c79aa0/09_param_rot_scale_view/src/three_app.js#L342


				const clip = THREE.AnimationClip.findByName(gltf.animations,clip_name);
				if(typeof clip.name == "undefined"){console.error(`clip ${clip_name} does not exist`);}
				console.log(`Mesh '${obj.name}' has animation '${clip.name}'`);
				const obj_mixer = new THREE.AnimationMixer(obj);
				const animAction = obj_mixer.clipAction(clip);
				animAction.play();
				obj_mixer.setTime(0);
				anim_params[obj.name] = {};
				anim_params[obj.name]["type"] = "mixer";
				anim_params[obj.name][clip_name] = {};
				anim_params[obj.name][clip_name]["mixer"] = obj_mixer;
				anim_params[obj.name][clip_name]["duration"] = clip.duration;
			}
		}
	});
	
}

function load_scene(user_on_load,gltf_filename){
	var loader = new THREE.GLTFLoader();
	loader.load(gltf_filename,

note the usage there of the variable clip_name that is used to find the animation by its name few lines above with THREE.AnimationClip.findByName(gltf.animations,clip_name);

And here comes the trick on how to find the clip name. I could not figure out a way out of the gltf only, so what I did to keep all the config in one place from Blender is the fallback on userData.parameter which has to be configured with the animation name as I’ve shown in the example above, or here it is again for convenience

Of course, you could simply hardcode it or use any other method to link the animation to your object, but that is the cleanest way I could come up with.

As you have noted, I’m using a structure anim_params to keep information about animations so that I can address it when I get an animation event request.

Nothing prevents an object from having multiple animations, so once you master all the steps to have one animation run, you’ll manage to extend this to multiple animations. You could place their names in the custom properties separated with comma or split them on multiple names from within the custom properties, or even better if you manage to extract the animation names from the GLTF without having to note them down a second time with custom properties.

Also I understand that your current main project does not have animations, there are no hurries, but once you have a branch or separate version or minimal example with animations integrated that might get clearer to you and probably easier to review and check if something is missing, but one step at a time.

An additional hint, the art of programming is not the art of writing code, it’s the art of reading code most of all

phil_crowther · June 10, 2020, 10:00pm

I had the feeling that we were headed back to where you started, but I think we have now narrowed things down quite a bit.

Here is the aircraft in a stripped down viewer:
Animation Viewer
(I couldn’t get rid of the skybox - a challenge for another day. If you press either mouse button, you can orbit around the aircraft.)

I will add the steps you suggest and see if that fixes the problems. It does sound promising.

I was looking at some of the 3JS examples with multi-animations and realized that my understanding of the mixer was probably not correct, since the examples had different mixers for different animations.

Yes, over the decades, I have written programs in several different languages: Fortran, Basic, assembly languages for 6502 (Apple) and 68000 (Amiga), and, more recently, C+.and js. In each case, I was pursuing a specific project and most of what I learned involved trying to interpret code that others had written and adapting it to my needs.

It’s not often that I have the author of the code to help me along. So I really appreciate your guidance.

phil_crowther · June 11, 2020, 4:24am

Got it!

You can see the results at:
Animation Viewer
Press the arrow keys to see the elevator and rudder move.

For the record, here are the changes I made:

At the beginning of the program, I inserted the following global variables:

var mixer1 = 0;
var mixer2 = 0;

In the function(gltf) section of the file loader, I inserted the following lines:

// The elevator animation
var clip1 = gltf.animations[0];
mixer1 = new THREE.AnimationMixer(object);
var action1 = mixer1.clipAction(clip1);
action1.play();
// The rudder animation
var clip2 = gltf.animations[1];
mixer2 = new THREE.AnimationMixer(object);
var action2 = mixer2.clipAction(clip2);
action2.play();

In the main program, I inserted the following commands:

if (mixer1) mixer1.setTime(elvatr/anmfps);
if (mixer2) mixer2.setTime(rudder/anmfps);

where elvatr and rudder have values of between 1 and 59 (since the animation has 60 steps) and anmfps = 24 (which is the default fps for Blender animations).

You need a separate mixer for the animation associated with each part because you will be using the mixer.setTime(t) command to set the position for each part.

I just need to test this in my flight simulator demo to make sure that the parts do not “detach” (I don’t see why they would) and that there is no huge frame rate hit.

Thanks for all your help and encouragement!

EDIT: I tested it out on the program and it works great! I only changed the movement of the elevator because it is relatively straightforward. The rudder movement will be more complex. And I need to add animations for 2 to 4 ailerons. And now that I know how to add animations, I can add all sorts of eye candy to the simulation, such as waving flags and people.

wassfila · June 11, 2020, 6:43am

Congratulations, I was sure it was just a matter of time.

Quite an impressive set of languages you have on your background. Make sure you keep the assembly actual as soon we’ll converge to a webassembly tendency which is not only about assembly but compiling from a multitude of other languages.

Back to your demo, I think from here on you can improve it the way you want as your objective of this thread is reached.

When it comes to the mixer global variables you added, that’s exactly what I did with the anim_params to the difference that in case you’d like to scale this up to many objects animations and not only one, you’ll need to store variables for each object and that’s all what anim_params is, a map to contain those variables for each object that is created dynamically. It’s good if you manage a design where you don’t have to modify anything on the animation js module for any new object, but that is up to you application scalability.

As a conclusion

I’m glad the concept of using an animation for a predictable movement of user input has helped
I see room for improvements here in three.js as such helper does not exist

phil_crowther · June 11, 2020, 7:58am

It certainly is something that could have a lot of interesting applications. For example, you could create a model car that is steerable, with opening doors and a movable suspension. Or perhaps create a sailing game where you could adjust the position of the sails to catch the wind.

I have contacted some Blender bloggers to see if they would like to share this with the Blender community. An increased interest could encourage the people at three.js to create a helper.

Three.js could also benefit from including your program in their examples since it shows how to do many unique things.

Thanks again for all your help!

wassfila · June 11, 2020, 9:02am

That’d be great, if you have any links to blender community interest let me know, I did not explore that yet,
I created a post in this forum to present the concept in a more generic way, as you can see the parameters do not only include animation as in your use case but many others such as colors, light, connection to external events such as MQTT and more

And I made a post about my complete Showcase example of a smar home application using that

I also know someone who’s expert in plane models construction and flight simulation programs who could test any demos you come up with.