Matrix4 rotation reverse

Hi, I tried to use the mediapipe to control my model, I applied the matrix4d from mediapipe to my model. The position and size transform works well but the rotation does not. It will rotate the opposite way if I Heads up/down or shake my head. But it works fine when tilting the head. How can I fix it on matrix4d or quaternion?
Here is my setting and loading code:

// camera
    this.camera = new THREE.PerspectiveCamera(
      30,
      this.windowResolution.width / this.windowResolution.height,
      0.1,
      20
    );
    this.camera.position.set(0, 0, -0.9);
    this.camera.lookAt(0, 0, 0);
...

// Load file asynchronously
  public async loadFile(path: string) {
    this.gltf = await this.loader.loadAsync(path);

    const vrm = await VRM.from(this.gltf); 
    this.vrm = vrm;
    this.faceModel = vrm.scene;

    if (this.faceModel) {
      const box = new THREE.Box3().setFromObject(this.faceModel);

      // Rescale the object to normalized space
      const size = box.getSize(new THREE.Vector3());
      const center = box.getCenter(new THREE.Vector3());
      box.setFromObject(this.faceModel);
      box.getCenter(center);
      box.getSize(size);
      this.faceModel.position.x += this.faceModel.position.x - center.x;
      this.faceModel.position.y += this.faceModel.position.y - center.y;
      this.faceModel.position.z += this.faceModel.position.z - center.z;


      // register default world center
      if (this.vrm.humanoid) {
        const initPosition =
          this.vrm.humanoid!.humanBones.hips[0].node.position;
          this.positionDefaultX = initPosition.x;
          this.positionDefaultY = initPosition.y;
          this.positionDefaultZ = initPosition.z;
      }
  }

I’ve had this before but I forget how I solved it.

It might be that the rotation part of the matrix needs to be transposed…

Or you may need to flip some axes…

https://threejs.org/docs/#api/en/math/Matrix4.transpose