How do you unscale a matrix?

I suggest you use Matrix4.decompose() to decompose a given matrix into its position, rotation and scale components. You then set the scale vector to (1,1,1) and use Matrix4.compose() to setup a new matrix.