But @donmccurdy gltf viewer does just fine with envmap and no lights, why? What is the magic setting?
So far the only major difference is that there (as well as here) PMREMGenerator is used - if that is the explanation, then I guess the darkness is basically a bug?
edit: aaaand yes, that was it. Environment maps other than what PMREMGenerator generates are just not useable