Vector3 rotation influenced by parent of parent

I have the following function trying to rotate the last joint of a finger using mediapipe results. However, the joint rotates even when the actual joint has remained straight, but one other in its parents chain has moved. Any ideas on how to combat this?

function updateHand(results) {
    let landmarks = results.landmarks[0];

    let localVector7 = new THREE.Vector3(landmarks[7].x, - landmarks[7].y, - landmarks[7].z);
    let localVector8 = new THREE.Vector3(landmarks[8].x, - landmarks[8].y, - landmarks[8].z);

    localVector7 = localVector7.normalize();

    let vector8relativeTo7 = new THREE.Vector3().subVectors(localVector8, localVector7);
    vector8relativeTo7 = vector8relativeTo7.normalize();

    let quaternion = new THREE.Quaternion();
    quaternion.setFromUnitVectors(hand.index[2].position, vector8relativeTo7);
    hand.index[2].setRotationFromQuaternion(quaternion);

    console.log("vector8relativeTo7: ", vector8relativeTo7);
}

The mediapipe output is not hierarchical is it? It’s a bunch of points in world space, so if you’re trying to compute angles from that data for joints on a animated mesh/skeleton, you’ll have to convert points into bone local space for that rig to do your inverse kinematics…
You can use the object3d methods worldToLocal and localToWorld to accomplish this…

like:
let localPosition = bone.worldToLocal( someDataPoint.clone() )
now localPosition will be in bone space and you can use it for .lookAt or angle determination…

Thanks for the quick reply.
My initial approach to the problem was using IK (and CCDIK) but it led nowhere.
This approach has looked more promising so far, and the thought behind to get the point-to-rotate-to in relation to the rotation point and, from what I understand, using the quaternion, setting the origin axes as the parent axes to figure out the resulting rotation.

From my previous attempts and research this attempt is closer to what you suggest but, still does not seem suitable for this case.

Yeah setting up IK is an art.
I’ve implemented the thing you’re talking about for full body avatar tracking and it took a lot of tweaking to get a decent rig.
You have to start out by figuring out the bones in hierarchy order, like first the hips, then the spine, shoulders/thighs, elbows/knees, because each level of the hierarchy depends on the orientations of the bones higher up.
I also had to perform some heavy handed filtering on the raw data coming from mediapipe/blazepose to get rid of the jitter.
Got it working pretty well in the end, but it was tricky.

1 Like