Should yield me something other than the 90 degree rotation. I have different matrices with different values but setFromRotationMatrix() seems to always return the same quaternion.

Those basis vectors in your example there look very close to an identityâ€¦ i.e.
0,1,0,
1,0,0
0,0,1
which looks very much like a 90 degree rotation of Y onto X, so 90 degrees around Z

So I suspect your data. Perhaps also a row/column swap? but yeahâ€¦

In your case, try conversion matrix â†’ euler â†’ quaternion:

const euler = new THREE.Euler().setFromRotationMatrix(rotationMatrix);
const quaternion = new THREE.Quaternion().setFromEuler(euler);

However, to be honest, the matrix does not contain pure rotation (or has XY swapped). The matrix looks like rotation around Z axis, but in this case it should be something like:

cos -sin 0 0
sin cos 0 0
0 0 1 0
0 0 0 1

so, the diagonal two values must be the same, like this:

This may be very basic, but my expectation was that any three orthogonal vector will give me a rotation. It may not be the rotation that I expect but it will be a rotation. However, thatâ€™s not true right? I could set them up in such a way that it has a rotation and a scale in say negative x, in which case setFromRotationMatrix fails because, well, itâ€™s not a rotation matrix?