How do I switch scene, clear up memory, and keep only the renderer scene and camera?

You can traverse through the scene, remove all objects and dispose the respective geometries, material and textures. Example code for this looks like so: