Replacing a material with MeshPhysicalMaterial with normalMap is not working

I am loading a GLTF model. I then need to replace the material of some of the meshes with a MeshPhysicalMaterial (MPM). This MPM material has a normalMap from a texture that I previously loaded. When replacing the material all the meshes with the new material are just black. If I replace the material but not add the normalMap it works OK.

If I add the MPM material on a new object it also works as expected. I guess I need to mark something as needing update but I am not sure what.

I have tried the following:

material.vertexColors = parseInt(material.vertexColors); //Ensure number
material.side = parseInt(material.side); //Ensure number
material.needsUpdate = true;
geometry.attributes.position.needsUpdate = true;
geometry.attributes.normal.needsUpdate = true;
if (geometry.attributes.color) {
  geometry.attributes.color.needsUpdate = true;
}
geometry.verticesNeedUpdate = true;
geometry.elementsNeedUpdate = true;
geometry.morphTargetsNeedUpdate = true;
geometry.uvsNeedUpdate = true;
geometry.normalsNeedUpdate = true;
geometry.colorsNeedUpdate = true;
geometry.tangentsNeedUpdate = true;

But nothing seems to work. Any ideas would be very much appreciated.

You don’t need to mark everything with needsUpdate if the geometry/material wasn’t rendered before. Do you assign the material or just the normalMap to their material?

It helps if you could create a codepen/fiddle that illustrates this issue.

/cc

I am replacing the whole material. I will try to create a codepen demonstrating this, not that easy to simulate though, but let me see what I can come up with.
Thanks.