Exporting PMREMGenerator output to .bin files?

Questions
, , ,
1

Howdy, my team uses Three version 129 and I’m exploring the possibility of moving to 147.

We have an internal tool which loads equirectangular .hdr files, runs them through PMREMGenerator, and then saves the output to a .bin file. The advantage of doing so is that we can load these pre-processed PMREMs into e.g. an Oculus Quest headset and sidestep the expensive conversion (which tends to fully lock up the headset for about half a second)

I’m migrating to version 147 now, and the PMREM system has changed considerably. The biggest changes causing problems for me seem to be:
-PMREMs can now be sizes other than 768x768
-PMREMs are half floats internally
-Images assigned as envmap are now automatically converted to PMREM (this is great, but not when I’m trying to get in the middle of that process and convert the image myself)
-my dataTextures simply… are not working

What worked in v129, approximately:

//Generate PMREM
hdrEquirecRenderTarget = pmremGenerator.fromEquirectangular( texture );    //

//Read PMREM into image buffer
imgBuffer = new Uint8Array( 2359296 );
renderer.readRenderTargetPixels ( hdrEquirecRenderTarget, 0, 0, 768, 768, imgBuffer )

//Put the texture into a blob for saving
var envBlob = new Blob([imgBuffer], { type: 'application/octet-stream' });
//<SAVE envBlob TO DISK>;

//To load the texture:
imgBuffer = new Uint8Array(<LOADED FILE>);
var loadedTexture = new THREE.DataTexture( imgBuffer, 768, 768, THREE.RGBAformat, THREE.UnsignedByteType );
loadedTexture.encoding = THREE.RGBEEncoding;
loadedTexture.mapping = THREE.CubeUVReflectionMapping;

//Assign to a mesh:
testMesh.material.envMap = loadedTexture;
testMesh.material.envMap.needsUpdate = true;

And I’m surely doing a lot wrong but… I can’t get any of this to work in 147. Here’s what I have:

//Generate PMREM
hdrEquirecRenderTarget = pmremGenerator.fromEquirectangular( texture );

//Read PMREM into image buffer
imgBuffer = new Uint16Array( hdrEquirecRenderTarget.width * hdrEquirecRenderTarget.height * 4 );
renderer.readRenderTargetPixels ( hdrEquirecRenderTarget, 0, 0, hdrEquirecRenderTarget.width, hdrEquirecRenderTarget.height, imgBuffer )

//Put the texture into a blob for saving
var envBlob = new Blob([imgBuffer], { type: 'application/octet-stream' });
//<SAVE envBlob TO DISK>;

//To load the texture:
imgBuffer = new Uint16Array(<LOADED FILE>);
var loadedTexture = new THREE.DataTexture( imgBuffer, hdrEquirecRenderTarget.width, hdrEquirecRenderTarget.height, THREE.RGBAformat, THREE.UnsignedByteType );

//Assign to a mesh:
testMesh.material.envMap = loadedTexture;
testMesh.material.envMap.needsUpdate = true;

And this doesn’t work even a little bit! As best I can figure, my dataTextures are totally malformed… but really, if anybody has any suggestions, I’m all ears. I know this is a fairly complicated process and I’ve had to cut my code down a lot to make this post manageable, but I’ll answer any questions that come up.

Thanks!

1 Like
2

Now that PMREM is handled internally, I don’t think you have to worry about it on application level anymore. Why don’t you just save and restore the HDR file instead?

PMREM is just a preprocessing step for the shader. Not something apps should be worried about. That’s why the generator was integrated in the renderer. If you write a PBR 3D engine from scratch, you wouldn’t expose PMREM to users in the first place. three.js did it because we couldn’t figure out an integration when the generator was introduced.

The PMREM data are also not intended for saving/loading. The implementation can change at any time and there is no guarantee that saved files can be loaded at a later point.

3 Likes
3

But this doesn’t solve the original issue where they want to avoid the preprocessing step. This sounds more akin to compressed textures. You’re doing all the preprocessing that you can so that you could put it on the gpu as raw bytes.

1 Like
4

Yeah, as Dubois says, we arrived at this solution because the PMREM process is costly, especially on mobile hardware like standalone VR headsets use. Before switching to preprocessed .bin files, we’d get half-second lockups every time a new envmap was brought in and that was before whichever ThreeJS version removed the 768x768 size limit.

It’s not a big deal to us if the PMREM standard changes again in the future since we don’t update our ThreeJS baseline often. It just means we’ll have to fix it every other year or so. (Which we’ve already had to in the jump from 112 to 129, as I recall)

So any insight into the current format of PMREMs and what I’m doing wrong with my buffers, encodings, etcetera?

5

Use THREE.HalfFloatType. THREE.UnsignedByteType was for THREE.RGBEEncoding which has been removed. Also make sure your data texture uses THREE.LinearFilter (the default is THREE.NearestFilter which isn’t right in this context).

2 Likes
6

Helpful tips and I thank you for them, but the real issue was…

image
image892×60 3.62 KB

I should have been more thorough with the Migration Guide. 135 → 136.

Got it working perfect now.

2 Likes