Rotate object so it always looks at specified Vectors

I grab vertex positions of characters hand during FBX animation, how can I rotate weapon so it always stays rotated relative to chars hand? Whatever I do I always end with implementation that either ignores hand’s rotation during animation, or ignores characters rotation when rotated on X or Z axis, however Y doesn’t break it. What I want is to take these points marked on picture and take their positions as directions in which I want every rotation axis to point at, and copy that rotation calculated based on positions of hand vertex onto weapon. How can I achieve this?

You could make a quaternion that rotates one coordinate system to another. The first coordinate system (cs1) is the initial position of your weapon, say in the holster, z axis pointing down along the leg, the second coordinate system (cs2) matches your hand, so that z vector is pointing out along the fingers; cs2 (and the quaternion) will change every frame of hand animation.

Then you calculate and apply this quaternion to the weapon and it will match the hand.

1 Like

If I understood right, you wrote this

CS1 = intial rotation of axe, Z axis is pointing along heros leg
CS2 = rotation of heros hand, Z axis is pointing along heros finger

My problem is I haven’t yet got to that part of the problem where I have to rotate
CS1 quaternion into CS2 quaternion, my current problem is one step back. Problem is
I don’t have CS2 quaternion, it doesn’t exist. I need to somehow create that CS2
quaternion and the only thing I have are these Vector3 position’s of vertexes. So
that is my current problem how do I create that CS2 quaternion that describes characters hand rotation based on Vector3 positions.

You can create a CS from the chosen points (-x, +x, -y, +y), see cs function.
Then you create a quaternion that rotates a mesh from world to the constructed CS, see csQuatern.
After that you can align meshes to that CS by assigning their quaternion to the one in the previous step.

1 Like

I just implemented your example and I still face the same problem I was facing with my previous attempts, this CS2 quaternion that I got from calculating vertex postions ignores character rotation, if I try to multiply CS2 with character quaternion I end up with rotation of axe that follows character rotation but ignores hand rotation during animation, just plain CS2 quaternion without multiplication with character quaternion gives good rotation on hand even during animation but rotation breaks if character gets rotated. Can I somehow combine character rotation with this CS2 quaternion? If I could somehow achieve that I will have a proper solution for this problem. Anyways big thanks for investing your time into making this good example.

You can get a quaternion of a mesh that accounts for all rotations, including the mesh’s parents via getWorldQuaternion:
https://threejs.org/docs/index.html?q=mesh#api/en/core/Object3D.getWorldQuaternion

So if your hand is a mesh somewhere inside the character parent mesh, you get the hand world quaternion and then apply it to the original positions of vertices (-x, +x, -y, y). This will give you the current world position of those vertices, including all rotations. From there you can build the coordinate system out of them and it’s quaternion.

1 Like

I don’t have parent child relation between character parent mesh and hand child mesh, because entire model is one mesh. I will try to implement what you said once I manage to wrap my head around it. So far applying characters world quaternion to hand vertexes only gave me even worse result, I am using this function from stackowerflow for getting hand vertex position during animation, and then with return of this function I use hero.localToWorld and then these new world vertexes use for logic you gave me in jsfiddle example. So to summarize my logic right now is vertex from hero → function transformedSkinVertex → hero.localToWorld → applyQuaternion(hero.worldQuaternion) → your jsFiddle logic → and apply quaternion from your logic onto my weapon model. I will try to figure out what am I doing wrong here.

const transformedSkinVertex = (skin, index) => {
    let skinIndices = (new THREE.Vector4()).fromBufferAttribute(skin.geometry.attributes.skinIndex, index);
    let skinWeights = (new THREE.Vector4()).fromBufferAttribute(skin.geometry.attributes.skinWeight, index);
    let skinVertex = (new THREE.Vector3()).fromBufferAttribute(skin.geometry.attributes.position, index).applyMatrix4(skin.bindMatrix);

    let result = new THREE.Vector3();
    let temp = new THREE.Vector3();
    let tempMatrix = new THREE.Matrix4();
    let properties = ['x', 'y', 'z', 'w'];
    
    for (let i = 0; i < 4; i++) {
        let boneIndex = skinIndices[properties[i]];
        tempMatrix.multiplyMatrices(skin.skeleton.bones[boneIndex].matrixWorld, skin.skeleton.boneInverses[boneIndex]);
        result.add(temp.copy(skinVertex).applyMatrix4(tempMatrix).multiplyScalar(skinWeights[properties[i]]));
    }

    return result.applyMatrix4(skin.bindMatrixInverse);
};

I was playing around with code from your example and with tiny change I got closer than ever before to solving this issue, just one thing remains. What I finally solved now is that with and without fbx animation playing, no matter what characters rotation is everything I need is covered now except for Z axis. What I changed in your code

original

const match = () => {
  mgun.quaternion.copy(mhand.quaternion);
  mgun.position.copy(hand.center);
  render();
};

my edit

const match = () => {
    const qh = csQuatern(hand_cs);

    mgun.quaternion.copy(qh);
    mgun.position.copy(hand.center);

    weaponObject.quaternion.copy(qh);
    weaponObject.position.copy(hand.center);
};

What I changed is in match() function instead of using “mhand.quaternion” I used “qh” the same quaternion you use in your align() function. The only problem is that rotation is wrong on Z axis, I assume by looking at your code that maybe Z axis should be specified where is Z+ looking at and where is Z- looking at? I assume that because in your cs() and csQuatern() function you only take into consideration X+, X-, Y+ and Y- axis. Is it possible to extend somehow this logic so when calculating “qh” quaternion to take into consideration directions of Z+ and Z- axis as well? To me it seems like the last step required for finally solving this problem. Perhaps the photo I posted wasn’t specific enough because I didn’t draw Z- and Z+ axis only one Z point, but If I could somehow take Z+ and Z- into consideration same way X+, X-, Y+ and Y- are taken.

The logic of making a coordinate system is this: points (-x, +x) define X axis.
Points (-y, +y) define a line _Y (not necessarily perpendicular to X) that, together with X, define a plane.
Z axis is perpendicular to the plane and is calculated as a cross product of X and _Y.
After that Y axis is calculated as a cross product of Z and X.
That should make a right-handed CS, so with points on your picture, Z should point to the left.

If you swap (-y, +y) points that should invert Z axis or apply Vector3.negate() from the library to it.

1 Like

I used negate() on X vector to get the end result I wanted

const cs = (p1, p2, p3, p4) => {
    const x = new THREE.Vector3().subVectors(p2, p1).negate();
    const _y = new THREE.Vector3().subVectors(p4, p3);
    const z = new THREE.Vector3().crossVectors(x, _y);
    const y = new THREE.Vector3().crossVectors(z, x);

    return [x.normalize(), y.normalize(), z.normalize()];
};

Now I can finally rotate weapon properly for my characters hand, huge thanks Tfoller. Since problem is now solved before I publish code and create a demo of all this I needed and you showed me how, there is one more question I assume everyone will need who searches for this topic. Since I now properly calculated CS2 quaternion and weapon is now properly rotated, can I somehow move weapons position along some axis of CS2 quaternion it uses? If axe can maintain its position and rotation and say move -1 from its current position on its CS2 quaternions Y axis? or any other axis. I had logic that did this but it doesn’t work with this new good implementation. I was doing something like this

const frontLocalVertex = transformedSkinVertex(charObj.model.children[meshIndex], weaponObj.frontVertice);
const backLocalVertex = transformedSkinVertex(charObj.model.children[meshIndex], weaponObj.backVertice);

const frontWorldVertex = charObj.model.children[meshIndex].localToWorld(new THREE.Vector3(frontLocalVertex.x, frontLocalVertex.y, frontLocalVertex.z));
const backWorldVertex = charObj.model.children[meshIndex].localToWorld(new THREE.Vector3(backLocalVertex.x, backLocalVertex.y, backLocalVertex.z));

// ignore these 5 lines of code, this is my old bad rotation
const finRotation = new THREE.Quaternion();
finRotation.setFromUnitVectors(frontWorldVertex, backWorldVertex);
finRotation.multiplyQuaternions(charObj.model.quaternion, finRotation);
weaponObj.quaternion.copy(finRotation);
weaponObj.lookAt(backWorldVertex);

// but position was good
const fixedPosition = new THREE.Vector3(weaponObj.vec[0], weaponObj.vec[1], weaponObj.vec[2]).applyQuaternion(finRotation);
weaponObj.position.copy(frontWorldVertex).add(fixedPosition.multiplyScalar(weaponObj.scalar));

So last question (for real now) is. How can I use Vector3 to change position of weapon relative to it’s current position while maintaining it’s CS2 quaternion, so for example Vector3(1, -1, 0.5) should move axe from its current position along CS2 quaternion axis. Assuming current position is treated as Vector3(0, 0, 0)

Edit: Ok this was simple actualy, all I had to do is after I copy position onto weapon to do

weaponObj.translateY(-0.1);

Problem solved, here’s the working code

// function taken from https://stackoverflow.com/questions/31620194/how-to-calculate-transformed-skin-vertices
// used to return positions of vertices from hero during animation
const transformedSkinVertex = (skin, index) => {
    let skinIndices = (new THREE.Vector4()).fromBufferAttribute(skin.geometry.attributes.skinIndex, index);
    let skinWeights = (new THREE.Vector4()).fromBufferAttribute(skin.geometry.attributes.skinWeight, index);
    let skinVertex = (new THREE.Vector3()).fromBufferAttribute(skin.geometry.attributes.position, index).applyMatrix4(skin.bindMatrix);

    let result = new THREE.Vector3();
    let temp = new THREE.Vector3();
    let tempMatrix = new THREE.Matrix4();
    let properties = ['x', 'y', 'z', 'w'];
    
    for (let i = 0; i < 4; i++) {
        let boneIndex = skinIndices[properties[i]];
        tempMatrix.multiplyMatrices(skin.skeleton.bones[boneIndex].matrixWorld, skin.skeleton.boneInverses[boneIndex]);
        result.add(temp.copy(skinVertex).applyMatrix4(tempMatrix).multiplyScalar(skinWeights[properties[i]]));
    }

    return result.applyMatrix4(skin.bindMatrixInverse);
};

// charObj.model - FBX object representing hero
const meshIndex = charObj.model.children.findIndex(e => e.type === 'SkinnedMesh');

// function taken from https://discourse.threejs.org/t/rotate-object-so-it-always-looks-at-specified-vectors/44130
const cs = (p1, p2, p3, p4) => {
    // I for my use need x axis negated
    const x = new THREE.Vector3().subVectors(p2, p1).negate();
    const _y = new THREE.Vector3().subVectors(p4, p3);
    const z = new THREE.Vector3().crossVectors(x, _y);
    const y = new THREE.Vector3().crossVectors(z, x);

    return [x.normalize(), y.normalize(), z.normalize()];
};

// function taken from https://discourse.threejs.org/t/rotate-object-so-it-always-looks-at-specified-vectors/44130
const csQuatern = cs => {
    const m4 = new THREE.Matrix4().makeBasis( 
        new THREE.Vector3(cs[0].x, cs[1].x, cs[2].x),
        new THREE.Vector3(cs[0].y, cs[1].y, cs[2].y),
        new THREE.Vector3(cs[0].z, cs[1].z, cs[2].z)
    );

    return new THREE.Quaternion().setFromRotationMatrix(m4).normalize().invert();
};

// local position of vertexes from hero's hand, positions of vertices are used as directions for weapon rotation
const xPositiveLocalVertex = transformedSkinVertex(charObj.model.children[meshIndex], 33105);
const xNegativeLocalVertex = transformedSkinVertex(charObj.model.children[meshIndex], 33108);
const zPositiveLocalVertex = transformedSkinVertex(charObj.model.children[meshIndex], 36411);
const zNegativeLocalVertex = transformedSkinVertex(charObj.model.children[meshIndex], 35802);
const yPositiveLocalVertex = transformedSkinVertex(charObj.model.children[meshIndex], 34566);
const yNegativeLocalVertex = transformedSkinVertex(charObj.model.children[meshIndex], 33981);

// world position of vertexes from hero's hand
const xPositiveWorldVertex = charObj.model.children[meshIndex].localToWorld(new THREE.Vector3(xPositiveLocalVertex.x, xPositiveLocalVertex.y, xPositiveLocalVertex.z));
const xNegativeWorldVertex = charObj.model.children[meshIndex].localToWorld(new THREE.Vector3(xNegativeLocalVertex.x, xNegativeLocalVertex.y, xNegativeLocalVertex.z));
const zPositiveWorldVertex = charObj.model.children[meshIndex].localToWorld(new THREE.Vector3(zPositiveLocalVertex.x, zPositiveLocalVertex.y, zPositiveLocalVertex.z));
const zNegativeWorldVertex = charObj.model.children[meshIndex].localToWorld(new THREE.Vector3(zNegativeLocalVertex.x, zNegativeLocalVertex.y, zNegativeLocalVertex.z));
const yPositiveWorldVertex = charObj.model.children[meshIndex].localToWorld(new THREE.Vector3(yPositiveLocalVertex.x, yPositiveLocalVertex.y, yPositiveLocalVertex.z));
const yNegativeWorldVertex = charObj.model.children[meshIndex].localToWorld(new THREE.Vector3(yNegativeLocalVertex.x, yNegativeLocalVertex.y, yNegativeLocalVertex.z));

const coords = {
    '-x': xNegativeWorldVertex,
    '+x': xPositiveWorldVertex,
    '-y': yNegativeWorldVertex,
    '+y': yPositiveWorldVertex,
    'center': zPositiveWorldVertex
};

const coords_cs = cs(coords['-x'], coords['+x'], coords['-y'], coords['+y']);
const qh = csQuatern(coords_cs);

weaponObj.quaternion.copy(qh);
weaponObj.position.copy(coords.center);          
weaponObj.translateY(-0.1); // I want weapon a bit lower positioned on Y axis of qh quaternion