How does Google know the angle of the location under the cursor,

Questions
, , ,
1

I was trying to understand how Google Street View knows where walls and the ground are.

I ran ai assistance in DevTool to find out, and was told Google sends hidden depth data with every panorama.

I vibe coded a repository to play around with this offline :

:backhand_index_pointing_right: My Repo: jobjojo/streetview-3d-render

Right now, I have a Three.js displaying the environment.

But I am stuck on the replication part.

  • How does the mouse cursor in Street View detect the wall and ground using these depth planes?
  • How do I actually code that math to make my cursor mesh tilt flat against the surfaces just like Google does?

Any tips or code help would be awesome.

BTW not much of coder or anyting.

Just wanted to know for sake of knowing.

2

I may be wrong but Google are likely using their 3D photogrammetry reconstruction of an environment to either generate depth maps or generate primitive boundaries from the constructed 3D data

3

if you have already generated planes from extrapolated 3D data you should be able to use the planes normal direction relative to the camera position to orient a marker