Making house floor panaroma image to 3D floor plan converter

I am trying to do something like this

So i created a sphere with a panaroma texture.
this is what i have done.
I am using plane buffer geometry.
I am creating plane geometry from corner vertices which i got from raycast.

Getting texture coordinates for material plane from 3d coordinate I convert it to uv point. And Apply same panaroma material over plane.

I am able to create floor plan but its not good.
I want to improve this.

I need help here in understanding.
How can i achieve that.
What i have done is right or not.