How to load gltf modules in React using GLTFLoader?

Yes, that should be the right place. Also see: Unable to Download GLB file with GLTFLoader

You do it like so (assuming the asset is in the public folder):

const loader = new GLTFLoader();
loader.load( './myModel.glb', gltf => {

 scene.add( gltf.scene );

} );