Matrix4 rotation reverse

Hi, I tried to use the mediapipe to control my model, I applied the matrix4d from mediapipe to my model. The position and size transform works well but the rotation does not. It will rotate the opposite way if I Heads up/down or shake my head. But it works fine when tilting the head. How can I fix it on matrix4d or quaternion?
Here is my setting and loading code:

// camera = new THREE.PerspectiveCamera(
      this.windowResolution.width / this.windowResolution.height,
    );, 0, -0.9);, 0, 0);

// Load file asynchronously
  public async loadFile(path: string) {
    this.gltf = await this.loader.loadAsync(path);

    const vrm = await VRM.from(this.gltf); 
    this.vrm = vrm;
    this.faceModel = vrm.scene;

    if (this.faceModel) {
      const box = new THREE.Box3().setFromObject(this.faceModel);

      // Rescale the object to normalized space
      const size = box.getSize(new THREE.Vector3());
      const center = box.getCenter(new THREE.Vector3());
      this.faceModel.position.x += this.faceModel.position.x - center.x;
      this.faceModel.position.y += this.faceModel.position.y - center.y;
      this.faceModel.position.z += this.faceModel.position.z - center.z;

      // register default world center
      if (this.vrm.humanoid) {
        const initPosition =
          this.positionDefaultX = initPosition.x;
          this.positionDefaultY = initPosition.y;
          this.positionDefaultZ = initPosition.z;

I’ve had this before but I forget how I solved it.

It might be that the rotation part of the matrix needs to be transposed…

Or you may need to flip some axes…