How to use DRACOLoader with GLTFLoader with bare module imports?

I’m working on a package that depends on three.js (specifically 3dTilesRendererJS) and it needs to load GLTF files itself, some of which depend on DRACO compression.

My question is – is there a recommended process by which I can include DRACOLoader in an external package that uses bare module imports to load three.js (thereby requiring import maps or build process to use)?

I can hack it into the package like so, but of course I can’t distribute it this way:

dracoLoader.setDecoderPath( 'https://unpkg.com/three@0.114.0/examples/js/libs/draco/' );

Given that this will be used in another package and there’s no known path ahead of time I’d like to keep using the bare module import statements so it can be resolved by the containing application:

dracoLoader.setDecoderPath( 'three/examples/js/libs/draco/' );

Ideally I’d be able to use dynamic import statements which are now supported in browsers and understood by build processes. I’m imagining something like so:

dracoLoader.getDecoder = function () {

    return import( 'three/examples/jsm/libs/draco' );

};

I know wasm still pretty rough around the edges in terms of how to load packages but maybe someone has some experience here.

ping @donmccurdy

Thanks!

I’m not at all sure this is widely understood by build processes — maybe for application build processes, but I don’t know how you’d author a package so that the end-user’s application understands your package’s dynamic imports.

Moreover, the wrapper file is not an ES module, so you can’t exactly ‘import’ it dynamically or otherwise…

If you could have the Draco decoder bundled at application compile time, would that work? Or do you need it to be dynamic? There are a number of ways you could recompile the Draco decoder — as a plain JavaScript ES module, for instance, which would behave much more intuitively. But even then I think the end-user will need to provide a path to anything that can be dynamically loaded — no package dependency can guarantee something ends up on the user’s webserver at a particular path. :confused:

@donmccurdy

I think dynamic imports are relatively well supported by build processes, now. Rollup.js, Webpack, and Parcel.js all support them. Even node supports it. I think that this point it’s okay to use dynamic imports in packages you expect to be resolved by a build process, which I’m already resigned to because of using bare imports.

I don’t know how you’d author a package so that the end-user’s application understands your package’s dynamic imports.

I would plan to distribute the package with a line that says

return import( 'three/examples/jsm/libs/draco/module.js' );

just like I use a line like so to import GLTFLoader

import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader.js';

And rely on the users build package resolution process to load the package. That may be one of the above build processes or something like import maps.

If you could have the Draco decoder bundled at application compile time, would that work? Or do you need it to be dynamic?

Ideally it would be dynamic so it’s not loading it into the users application unless they try to load a model that depends on DRACO compression.

But even then I think the end-user will need to provide a path to anything that can be dynamically loaded — no package dependency can guarantee something ends up on the user’s webserver at a particular path. :confused:

I think the “fixed path” model that DRACOLoader is using right now with setDecoderPath would need to be rethought. The loading of the wasm package is the part I’m less familiar with, though, and whether or not anything special needs to happen. Maybe it’s worth making an issue in three.js to discuss how this could be updated? At the moment it’s difficult to use DRACOLoader with imports without asking users to do manual work.

@donmccurdy Digging a bit more I see that the draco decoder source is from this DRACO repo and looks particularly complex to regenerate. Maybe that DRACO repo is a better place to make a request for an es6 module version of the decoder. I’m not familiar enough with the details of emscripten to know would would go into that.

Maybe that DRACO repo is a better place to make a request for an es6 module version of the decoder.

I expect the Draco maintainers are less familiar with JS build tooling than we are, and I doubt they want to maintain that — especially when a pure-JS build is going to be slower than the WASM equivalent.

I think that this point it’s okay to use dynamic imports in packages you expect to be resolved by a build process…

I can’t imagine how this would work without help from the end-user, with current build tools… Maybe in Parcel.js, since it’s much more opinionated about use with applications? But I’ve never heard of a downstream dependency enabling code splitting all by itself. It’d be useful, for sure, but I have no idea how to do it. If it worked for WASM, couldn’t you distribute textures, audio files, and more on npm in the same way?

Honestly I see this is a huge ergonomic problem with WASM in general, and the JS community (devs of bundlers? npm? libraries?) need to come up with a coherent way to distribute WASM in packages, not just in final applications. If I knew how to solve this I would happily make the change to DRACOLoader. With Basis in the mix it’s pretty easy to imagine ending up with 3x the problem:

  • draco_decoder.js
  • draco_decoder.wasm
  • basis_transcoder.js
  • basis_transcoder.wasm
  • zstd_decoder.js
  • zstd_decoder.wasm

And add three more scripts if you want JS fallback for browsers that don’t support WASM. :confused:

It’s possible to make a .wasm file that doesn’t need a JS wrapper (standalone WASM), but I don’t think it’s possible in every case, and I don’t really understand how to use that.

I might not be following. Are you saying you’re not seeing how using dynamic import statements could work at all without user involvement or that it’s not clear how it would work with WASM specifically? In case it’s the former the documentation I linked to above all state that dynamic imports work. I plan to distribute the package as only an es6 module file with no build process ahead of time. Notionally when it’s used in a users code the users build process will decide to create a separate code chunk when it encounters a dynamic imports statement. I’m not sure why a user would have to do anything special to enable that. In the past it may have been a pain but there is now universal syntax for “load this when I need” with import(). I may have linked the wrong section for Rollup above but the docs mention that dynamic imports are part of the modules spec and looking through the issues in the repo there are issues mentioning it’s inclusion.

Note I haven’t tested it in all these build processes but I don’t know why it couldn’t work. Leaving dynamic imports aside, though, if the package could just be imported as a regular module so a bundler could handle it more easily that would satisfy me for now.

Honestly I see this is a huge ergonomic problem with WASM in general, and the JS community (devs of bundlers? npm? libraries?) need to come up with a coherent way to distribute WASM in packages, not just in final applications.

It really is. It would be great to just be able to import * from './path/to/package.wasm'. I think the thing I struggle with is finding a solution that meets proper web standards so it’s technically “real” javascript that bundlers can also understand and make “just work”. It happened for modules and imports so hopefully it’s just a matter of time for WASM. Honestly I think it’s one of the big things that’s hindering adoption for web assembly.

If I knew how to solve this I would happily make the change to DRACOLoader.

Is it absurd to think that base64 encoding the WASM data and embedding it in a .js file my be a viable option to enable imports? It might not be possible to do out of the box with emscripten but it could just be a simple post processing step to enable.

@donmccurdy

A quick experiment on string encoding the wasm data using TextDecoder to test file size:

fetch( 'https://unpkg.com/three/examples/js/libs/draco/draco_decoder.wasm' )
    .then( res => res.arrayBuffer() )
    .then( buffer => {

        const td = new TextDecoder();
        const str = td.decode( new Uint8Array( buffer ) );
        saveData( new Blob( [ str ] ), 'encodedWasm.js' );

    } ); 

“saveData” function from here. The original draco_decoder.wasm is 323kb large while the saved out encoded file is 341kb – just around a 5% increase. You’d have to tack on some quotes and an export statement but it doesn’t seem too bad.

I might not be following. Are you saying you’re not seeing how using dynamic import statements could work at all without user involvement or that it’s not clear how it would work with WASM specifically?

Set aside WASM for now, considering just a plain ES module. Let’s say I have a web application, I’m using some unspecified build process, and my web application depends on your package. There is nothing your package can do that will reliably cause my web application to deploy multiple JavaScript files if I haven’t configured my build and/or deployment process to expect that… I agree Parcel would work out of the box, but Rollup looks like it at least needs you to configure a directory for chunks to be written into. It would be nice if this “just worked” broadly, but I’ve never heard of a package pulling it off, and certainly it won’t work on a CDN today… Do you want to do a proof of concept, and then test a few bundlers to see what they do? Maybe if the user intervention is minimal enough it would still be an improvement on the current situation.

A quick experiment on string encoding the wasm data using TextDecoder to test file size…

This won’t roundtrip correctly with TextEncoder, because arbitrary binary data doesn’t have a 1:1 mapping to utf-8 characters. I think you’ll need base64, and I’d expect more like 20-30% inflation and some decoding time with that.

Screen Shot 2020-07-03 at 11.14.24 AM

@donmccurdy

There is nothing your package can do that will reliably cause my web application to deploy multiple JavaScript files if I haven’t configured my build and/or deployment process to expect that…

Thanks for your patience – I’ve tried this with the mentioned three bundlers and I think I see what you’re saying now. While they all support dynamic imports the the complexity seems to stem from the inability to properly resolve the path of the current script in order to load a dynamically loaded file relative to it. Webpack uses script tags to load a dynamic chunk but that script is loaded relative to the root html page rather than the script that’s referencing it which can cause problems.

Here’s what I found when testing out the three bundlers and you can find the set ups here if you want to check it out:

Rollup

This “just works” if you you specify a directory to output to rather than a file and output to “cjs” or “module” format. If “iife” or “umd” are specified or you try to output to a single file then the --inlineDynamicImports flag must be used. Rollup fails if it’s not used and informs the user in a command line warning.

Webpack

This almost just works except for the above mentioned relative chunk path problem. By default if the root html page tries to load the index js file like so: ./bundle/index.js then the chunk loading will not work. In order to fix this you need to specify the output.publicPath option to be ./bundle/.

Parcel

This does just work. It properly loads the dynamic imports relative to the original script. Out of curiosity I dug into the source to see how they were resolving the current script path and it looks like they’re scraping the current stack trace get the current script path.

I’m not sure if there’s a reason that webpack isn’t using the stack trace method but it would definitely simplify things for the end user. At the least it could check if document.currentScript is supported first and fall back to the stack trace approach.

I’m not sure what you make of the rollup case but it seems like Webpack should be able to handle this correctly similar to parcel, right?

This won’t roundtrip correctly with TextEncoder, because arbitrary binary data doesn’t have a 1:1 mapping to utf-8 characters.

Well that’s disappointing. Using base64 I see a filesize of 431kb or around a 33% increase. Encoding to just a basic string I and skipping base64 I see a file size of 333kb or an increase of 3%. I’m not sure if there’s another reason to not do that, though. I suppose in that case we can’t rely on the browser to decode it asynchronously using a blob url.

fetch( 'https://unpkg.com/three/examples/js/libs/draco/draco_decoder.wasm' )
    .then( res => res.arrayBuffer() )
    .then( buffer => {

        const ua = new Uint8Array( buffer );
        let str = '';
        for ( let i = 0, l = ua.length; i < l; i ++ ) str += String.fromCharCode( ua[ i ] );
        saveData( new Blob( [ str ] ), 'raw-string.js' ); // 333kb

        const b64 = btoa( str );
        saveData( new Blob( [ b64 ] ), 'base64-string.js' ); // 431kb

    } );

@donmccurdy

If you’re interested I’ve made an issue in the webpack repo to see if this can be addressed in webpack. I’m not sure what you make of the current rollup behavior, though. It would be nice if UMD or IIFE supported dynamic chunks but it would also add a bit of complexity to rollup. At the least it does support in-lining the dynamic content pretty simply.

1 Like

Maybe the long-term answer is GitHub - WebAssembly/esm-integration: ECMAScript module integration. It would certainly be nice to be able to just import a .wasm file as easily as a .js file. Apparently you can test that in Node.js now.

Encoding to just a basic string I and skipping base64 I see a file size of 333kb or an increase of 3%. I’m not sure if there’s another reason to not do that, though.

I think this runs into the same problem as before — are you sure you can decode it afterward? Base64 is the general purpose string<->binary mapping, for better or worse.

Also, we should really figure out how to get rid of the JS wrapper objects and just use .wasm directly. Then there are only half as many files to juggle. Surely that is possible…

@donmccurdy

Maybe the long-term answer is https://github.com/WebAssembly/esm-integration . It would certainly be nice to be able to just import a .wasm file as easily as a .js file. Apparently you can test that in Node.js now .

That definitely sounds like the right way to do it in the long run. Unfortunately I can’t find a whole lot on the state of the feature online. Node seems to be doing it’s own work in that direction but have Google, Apple, Mozilla, etc stated their intent to implement?

I think this runs into the same problem as before — are you sure you can decode it afterward? Base64 is the general purpose string<->binary mapping, for better or worse.

I’d converted it back and made sure it matched the original array correctly but according to this stackoverflow answer and the base64 wikipedia article base64 is intended to ensure that data is transmitted correctly in some scenarios. So it seems that base64 is generally the best option even though the raw string seems okay locally.

Also, we should really figure out how to get rid of the JS wrapper objects and just use .wasm directly. Then there are only half as many files to juggle. Surely that is possible…

I’m not an expert in why emscripten exports the separate wrapper but if we can get this to work with modules using base64 that should address the problem in my opinion. The wrapper and binary should be able to pack into a single file or at the least that they can just be imported via modules should simplify things quite a bit.

I think if we’re comfortable using import() then we could make a module version of DracoLoader that automatically loads the wasm files only when needed and can imported in user applications without manual work (webpack issue pending). I’m not sure how to deal with with the fact that DracoLoader is still maintained as a non-module file right now, though.

Maybe for the moment we could bundle up module versions of the draco libraries and modify DracoLoader so the user could provide a dynamic import callback?

Here’s an example packing both together:

^it works well there because the WASM is quite small. Draco is not as compact, so the ~30% base64 penalty definitely hurts a bit more.

Maybe some useful stuff here:

1 Like

@donmccurdy

^it works well there because the WASM is quite small. Draco is not as compact, so the ~30% base64 penalty definitely hurts a bit more.

I actually think the apparent impact of the draco decoder filesize won’t be so bad in typical cases considering it can be downloaded in parallel with the model. For a better impression of the download sizes I tried GZipping both the original wasm file and base64 one I generated and saw file sizes of ~85kb and ~131kb respectively.

Just to walk through the parallel download thread a bit more if we look at the draco-compressed cesium-man gltf file it weighs in at 322kb and none of the file types are typically gzipped by default over the network I don’t think:

  1. If an app preemptively downloads the draco decompressor at the same time as the GLTF then it can load it in parallel as the gltf, bin, and jpg files are downloaded.

  2. If the app doesn’t preemptively load it and waits until a GLTF file arrives that requires draco decompression then at least the decompressor can be download along side the bin and the jpg files.

  3. The worst case is where the app waits for a GLTF file to arrive that requires draco but otherwise has all data embedded. I’m not sure how common that is but in that case you’d have to incur the serial cost of loading the GLTF file and then loading draco. Of course it’d only have to happen once and could be addressed by preemptively loading the decompressor if you know your files will require it.

I guess it feels like the model files that are being downloaded will generally dwarf the size of the draco wasm file in more complex cases and should be able to be downloaded in parallel anyway. And since we can make it dynamically loaded it shouldn’t affect the webpages time to first render.

Maybe some useful stuff here:

Sounds like everyone is feeling these growing pains… It doesn’t sound like base64 encoding the wasm data is an unpopular approach, though.

85kb → 131kb is adding 50kb, regardless of the timing details. It’s true that the size of the application’s JS/WASM code might not be the dominant factor in its time-to-first-render, but it’s still 50kb that exist only for developer ergonomics. Base64 makes sense for a really compact WASM blob (like meshopt’s decoder), but I don’t think +50kb for developer ergonomics is a good trade.

It’s also very likely that the Draco WASM library is not as compact as it could be, but I’m not the right person to optimize something like that. By comparison, the meshopt decoder is 20kb, before gzip.

@donmccurdy

I don’t think +50kb for developer ergonomics is a good trade.

Are you against even providing the option so developers can make their own decision? According to chromestatus it looks like there hasn’t been any real movement on wasm esm integration which isn’t all that hopeful.