Loading EXR embedded inside GLB

[Torben_Van_Assche]

How would I load the EXR file from the buffer directly?
Loading it from the buffer it appears to think its a texture.

var parser = gltf.parser;
var bufferPromises = parser.getDependency("bufferView", parser.json.images[2].bufferView);

bufferPromises.then(function (buffer) {
   var exr = that.assetLoader.parseEXR(buffer)
	console.log(exr)
   exr.mapping = THREE.EquirectangularReflectionMapping;
   that.exr = that.pmremGenerator.fromEquirectangular(exr).texture;
});

new nixps.cloudflow.visual.EXRLoader().load(that.options.exr, function(exr) {
   console.log(exr)
})

[Fennec]

It is a Texture! try the following:

bufferPromises.then(function (buffer) {
   ...
   that.exr = that.pmremGenerator.fromEquirectangular(exr).texture;
   myScene.background = that.exr;
   myScene.environment =  that.exr;
});

[Torben_Van_Assche]

Unfortunately that results in the following , which is very odd since I should be getting the same data if I parse or load the data, shouldn’t I?

[Fennec]

It seems like the EXRLoader’s parser and loader methods doesn’t return the same thing, with the loader you get a DataTexture, with the parser you get the asset data and you need to construct the DataTexture yourself, the following code should work

bufferPromises.then(function (buffer) {
   const { data, width, height, format, type } = that.assetLoader.parseEXR(buffer);

   // Create the DataTexture from the parsed EXR
   const dataTexture = new THREE.DataTexture(
      data, 
      width, 
      height, 
      format, 
      type, 
      THREE.EquirectangularReflectionMapping
   );

   // Generate the environment Texture with pmremGenerator
   const envTexture = that.pmremGenerator.fromEquirectangular(dataTexture).texture;

   // Use it with your scene
   myScene.background = dataTexture;
   myScene.environment =  envTexture;
});

[Torben_Van_Assche]

My current code is, which I have matched to your suggestion, but there doesnt appear to be anything in the resulting that.exr… I get a texture but the data inside it doesn’t do anything in my viewport.

var parser = gltf.parser;
var bufferPromises = parser.getDependency("bufferView", parser.json.images[2].bufferView);

bufferPromises.then(function (buffer) {
    var exr = new THREE.EXRLoader().parseEXR(buffer);
    var dataTexture = new THREE.DataTexture(exr.data, exr.width, exr.height, exr.format, exr.type, THREE.EquirectangularReflectionMapping)
    that.exr = that.pmremGenerator.fromEquirectangular(dataTexture).texture;
                            
    that.scene.background = that.exr;
    that.scene.environment = that.exr;

[Fennec]

For scene the background, change the following line (sorry! Untested code )
that.scene.background = dataTexture

@Torben_Van_Assche

it doesn’t do anything in my viewport

Do you have any reflective materials in your scene, something like:

let material = new THREE.MeshStandardMaterial( {
	metalness: 1,
	roughness: 0,
	envMapIntensity: 1.0
} );

You can refer to this example to see how it work

[donmccurdy]

@Fennec

It seems like the EXRLoader’s parser and loader methods doesn’t return the same thing, with the loader you get a DataTexture,

This sounds like something we should fix in the loader perhaps? Generally I think we want loaders to have ‘load’ and ‘parse’ methods that return the same result.

---

I’ll just note (e.g. for readers) that glTF does not officially support OpenEXR textures — you have a custom pipeline adding EXR textures to the GLB I imagine?

[Torben_Van_Assche]

We do yes, we are exporting from out own OpenGL editor to view with threejs and want to embed all our data into a single fetch from the database.

[Fennec]

@donmccurdy

This sounds like something we should fix in the loader perhaps?

I do agree, it can create some confusion, like in this particular case. There is the backward compatibility problem, would this be considered as a breaking change?

[Torben_Van_Assche]

This seems to load, but the lighting doesnt look like it is being applied.
I have only the glb, a camera and an ambient light in my scene.
Is that enough or am I missing a directional light for light to behave properly?

EDIT: I don’t immediately see anything I am missing. this setup worked when I was using an HDR at the very least, now I just get no reflections at all

[Fennec]

Try adding a mesh with reflective material to your scene, if you can see the reflection on the mesh there is a problem with you model, if you can’t there is a problem with the EXR file.

const geometry = new THREE.SphereGeometry();
const material = new THREE.MeshStandardMaterial( {
	metalness: 1,
	roughness: 0,
	envMapIntensity: 1.0
} );

const sphereMesh = new THREE.Mesh( geometry, material );
scene.add(sphereMesh)

Adjust the mesh’s position and scale so it fit within your viewport.

[donmccurdy]

DataTexture is a subclass of Texture, and EXRLoader is a subclass of DataTextureLoader … so returning DataTexture consistently isn’t necessarily a breaking change I think. But there could be more reasons for all this, would have to review a PR I think.

---

Might also be worth trying something like TextureHelper to display the texture for testing purposes.

[Torben_Van_Assche]

I exported as a gltf and loaded it via the regular loading method, which works perfectly.
parsing it does not work even with the steps I perform here…

bufferPromises.then(function (buffer) {
      var exr = that.assetLoader.parseEXR(buffer);
      var dataTexture = new THREE.DataTexture(exr.data, exr.width, exr.height, exr.format, exr.type, THREE.EquirectangularReflectionMapping)
      var envMap = that.pmremGenerator.fromEquirectangular(dataTexture)
      that.exr = envMap.texture;
      that.scene.environment = envMap.texture
});

new nixps.cloudflow.visual.EXRLoader().load(that.options.exr, function (exr) {
      that.scene.background = exr;
      var envMap = that.pmremGenerator.fromEquirectangular(exr)
      that.scene.environment = envMap.texture;
      that.exr = envMap.texture;
})

[Torben_Van_Assche]

An alternative approach in me not relying on the DataTexture constructor does not seem to work either. The parsed data does not contain an image but does have the necesarry data (I think, its a Uint16Array(2097152)) which is rather large but makes sense in the context of this being a texture…

bufferPromises.then(function (buffer) {
     var texData = that.assetLoader.parseEXR(buffer);
     var texture = new THREE.DataTexture()
     console.log(texData);

     texture.image.data = texData.data;
     texture.source.data.width = texData.width;
     texture.source.data.height = texData.height;
             
     texture.format = texData.format;
     texture.type = texData.type;
     texture.colorSpace = THREE.LinearSRGBColorSpace;
     texture.minFilter = THREE.LinearFilter;
     texture.magFilter = THREE.LinearFilter;
     texture.generateMipmaps = false;
     texture.flipY = false;

     var envMap = that.pmremGenerator.fromEquirectangular(texture)

     console.log(texture)

     that.exr = envMap.texture;
     that.scene.environment = envMap;
     that.scene.background = envMap.texture;
});