Making house floor panaroma image to 3D floor plan converter

I am trying to do something like this

So i created a sphere with a panaroma texture.
this is what i have done.

I am using plane buffer geometry.
I am creating plane geometry from corner vertices which i got from raycast.
Getting texture coordinates for plane.

I am able to create floor plan but its not good.

The links which I am taking as references are using image processing and built on threejs only.
I need help here in understanding.
How can i achieve that.
What i have done is right or not.