MeshStandardMaterial lightmaps

Does anyone know if i can have lightmaps totally lighting a mesh, ignoring the envmap for lighting?
I still want the envmap for reflections and all the other benefits of a standard material.

At the moment it appears that the envmap is making my lightmap shadow areas lighter as its lighting that area of the mesh that the lightmap is trying to make darker.

To use an exampke, say I have a city street with buildings on both sides. One side has the building faces lit up and the other side hss the building faces in shade from a HDR
Now i want the windows of the buildings to refelect the other side of the street so I create a HDR with the buildings rendered into it (using blender and the original HDR)
So I now just want the buildings lit by a lightmap as the new HDR will light the buildings wrong. Effectively it’s lighting the buildings in the scene with the buildings in the HDR. So buildings in shade are being made brighter and the lit buildings made darker aa they are facing the opposite side of the street in the new HDR
Can I turn off the envmap lighting the mesh?