Porting Three.js to WebAssembly with AssemblyScript

Hello!

We are porting Three.js to AssemblyScript (a strict subset of TypeScript that compiles to WebAssembly).

:point_right: github.com/infamous/glas

The project is in the early stages, but we already have a number of classes ported over along with passing unit tests. For example here is Matrix3 and its unit tests.

Join us over in the repo or chat with us at gitter.im/infamous/glas.

5 Likes

I love your enthousiasm.
But, and this is a legitimate question, I really don’t know and Iam not implying anything, is it really as simple as just converting code?
Do you have access to the canvas context? To the WebGL APIs?

For most of the code, yes, it is as simple as converting code (including the unit tests), as you can see in the Matrix3 example I linked above.

At the moment, no: the WebAssembly context does not have direct access to the canvas. Once we get to porting WebGLRenderer, then instead of having a canvas reference inside of it, we will have a stack of render commands (a command queue) in the WASM context. With some glue code, we’ll send the stack of commands over to the JS side, and on the JS side there will be a handler that receives the commands (which contain shaders, attributes, etc) that will pass them directly to the GPU.

This will be the hardest part of the porting effort.

Similar things have already been done for Canvas 2D APIs. For example, @jtenner, an active contributor to AssemblyScript, and creator of the as-pect unit testing framework for AssemblyScript which we’re using for the unit tests in glas, created a project called as2d in which he demonstrated how to send commands from the WASM side to a canvas 2d context on the JS side. Now we’ll use a similar technique to render to a canvas webgl context.

It’s a process, and it will take some time to implement for sure.

Of course, the WebAssembly GC spec is underway, and at some point in the near future we will be able to reference DOM and JS objects from WebAssembly. The WebAssembly Reference Types spec is already released behind a flag in Google Chrome (I haven’t checked other browsers on this yet), which is a pre-requisite for WebAssembly GC, and finally DOM/JS interop (including importing JS modules into WASM, and vice versa).

It’s the future!

We can use all the help that we can get, if you’re interested!

4 Likes

Do you have any idea of performance of AssemblyScript vs C/C++/Rust -> Wasm?

I had a look but couldn’t find anything related to this.

My thinking on using Wasm with three.js has been, rather than converting everything to Wasm, we should figure out the hot paths and rewrite them in C/C++/Rust -> Wasm (or whatever language).

The maths library would be an obvious candidate for this. There may already be highly optimised math libraries that we could repurpose here.

Rewriting non-hot paths in wasm may give less of a performance boost than you expect. For example, if something takes 50ms in JS, vs 2ms in Wasm, but only runs a couple of times, then you are not gaining anything a user will notice and your effort is essentially wasted. As a result, I would concentrate on hot paths at first.

EDIT: I found this page (summarised here) comparing C/Rust/AssemblyScript and also giving a nice summary of replacing a hot path in squoosh.app with Wasm.

Surprisingly, AssemblyScript comes out very strong - nearly as performant as C, and a very small file size.That said, they were testing a single file so I would not consider this conclusive.

I still think that it would be worth adapting a pre-built and battle tested game math library written in C++ though. Simple converting JS->AssemblyScript->Wasm is unlikely to give the best possible performance.

1 Like

Here I played with AssemblyScript little bit for heavy calculations in loop:

And I realized that we are facing with different aspects when we want good performance right now.

First problem: that Wasm is sandboxed for security reasons and Wasm<->JS interoperation is pretty heavy. You need to calculate all stuff in one big wasm block in order to avoid it. WebAssembly not an assembler, and you can’t rewrite one small piece of code with it to get performance boost. You can make performance worse accidentally if you call wasm too much.

Second problem: sometimes you have unexpected results for different data structures in AssemblyScript (and probably in rust/c++, I am not try). For example when I return a class from my function I accidentally get object explosion problem and x10 performance drops that become worse than JS.

Also If you want to pass big array to/from wasm you should know best way how to do it.

Also recently TensorFlow created new wasm backend and they get x10 boost for float32 calculations on CPU.

All this is good but you should know! That probably on your machine In most of examples you have a bottleneck on GPU, not on CPU. At least I get such results. You can try to measure CPU/GPU load by yourself with this tool. For future we should think about WebGPU too, since we have bottleneck on a video-card.

That might the case for most mobile devices, but at least on my laptop it’s the opposite.

This is a very good point, but as far as I’m aware it’s something that the wasm team are working to reduce and should be less of an issue in the future.

This article of the wasm roadmap explains it well. It’s pretty big so skip to the Small modules interoperating with JavaScript section:

@trusktr the more I research on this, the more it seems to me like converting big libraries completely to wasm is not something that anybody is doing right now. And that includes huge companies that certainly have the resources to do so. Instead, they are converting single big tasks that can easily happen off the main thread and take a long time in JS, like image recognition.

Your project is certainly a valid project for experimentation and learning, but I’m not convinced the performance gain will be worth it. Otherwise, why aren’t other people doing it?

Of course, perhaps they are and I just haven’t heard about it!

3 Likes

Interesting

They have an example with parser, it’s pretty big block of code. I am talking about ~15 instructions in wasm module that you can call from js 3000 times in one frame. In another words: all loops need to be inside wasm sandbox, except requestAnimationFrame loop.

Here is a demo with FPS counter: https://jtiscione.github.io/webassembly-wave/index.html (as you saw in the other article you found, AssemblyScript is comparable to Clang and Emscripten).

AssemblyScript is using WebAssembly/binaryen's optimization pipeline (by Alan Zakai, same person who made Emscripten).

That may be one way to do it, but I’m aiming for a great end-developer experience (everything being in a single easy-to-use language like AssemblyScript).

Bullet (Ammo) in Three.js is a black box, and developers can not simply fork it and update things they need (for sake of example imagine someone needs to customize something in Bullet).

With a simple system in place for all code (f.e. AssemblyScript), the end user can easily fork any part of the code, and even be more likely to make contributions back to the project.

Replacing hot paths might be “enough”, but I don’t think it will lead to the developer experience that I’m imagining.

AssemblyScript now supports threads. The WebAssembly GC spec will give us the ability to reference “host objects” such as DOM and JS objects inside WASM is underway (this includes WASM modules directly importing ES Modules, and vice versa). With these advancements we’ll be able to make optimizations by avoiding the current cost of WASM-JS communication needed to access a canvas, and by placing different parts of the system on separate threads (:drooling_face: :drooling_face: :drooling_face:), all while maintaining a developer experience that web developers are familiar with.

AssemblyScript is simply easy to use and it works great with existing TypeScript tooling (VS Code Intellisense, Prettier, ESLint, Webpack, etc) as long as we don’t use the few features that are AS-specific, which is what we’re doing in glas.

This is a possibility. If we did this, I’d want to port the tool over to AS too, with the goal of having everything in the single-language and with the best end-dev experience.

This is true, but in Firefox they’ve already made huge improvements:

Calls between JavaScript and WebAssembly are finally fast :tada:

The WebAssembly GC features I linked above will allow us to totally bypass this problem when that comes out.

Yep! So the design in glas will be that all code runs in WASM (all code is AssemblyScript), and for every frame the WASM module will send the JS a single command queue with a single WASM-JS call, to minimize this issue for the time being.

If we were to replace only various hot paths with WASM, this would mean we’d need to call into WASM multiple times.

Interesting! Thanks for pointing that out. In glas we’re using only top-level classes and keeping it simple like Three.js (we’re not changing the structure except as needed to make it work in AS).

Nice! Thanks! That’ll come in handy once we get to WebGLRenderer.

This is true! It probably depends on the specific application (f.e. @looeee’s scene is more CPU heavy on his laptop, and use of a lot of physics would increase CPU, etc).

At work, one of the bottlenecks we have is rendering many of the same thing, and each individual mesh causes one expensive GPU call.

I have some plans with InstancedMesh to reduce the number of GPU calls by automatically instancing meshes in a subtree, so that we can use the scene graph tree the way we do now (organizing transforms in a hierarchy) instead of having to manage separate lists of instanced objects (this is spread across two issues, sorry):

I’ve imagined how to do it, I just need to implement it.

You can try to measure CPU/GPU load by yourself with this tool.

Niiiiiiiice! Thanks! That’s neat!


On a different note, WebAssembly can be compiled to native assembly which means that eventually we can have an OpenGL ES target, not just a WebGL target. Here are some tools to do that:


I guess I’m ambitious. :smiley:

Plus, if they already have big applications written in JS, and they have business needs, they perhaps don’t want to afford the cost.

But in my case, I’m doing it for free, because… hack yeah :computer::floppy_disk::zap:!

Because, it takes time and effort. Most people want to use what exists to make products. Most businesses use what exists so they can make money, not so they can invent a free tool that no one will pay them for. This is why, despite that WebGL is now in every browser, big companies still use 2D HTML for UIs, and no big companies write WebGL UI because they don’t need to, and game companies use existing native tech for games, because they don’t need to port to WebGL (and some tools compile to WebGL). When you look at business needs, this all makes sense.

When people take passion, and put business needs aside, then the best things come forward; things like Three.js, AssemblyScript, NeoVim text editor, and many other free projects that business-oriented organizations are less likely to create.

And then, once those amazing free projects are released into the wild, suddenly businesses realize they can use these tools for their own profit. Many companies started using Three.js, for example, but no companies wrote their own engine because, they didn’t need to. They’ve started using AssemblyScript, because they see its potential after its creation.

Well, companies don’t start using NeoVim, that’ll always be from passion for passion. But Before NeoVim, no company was willing to make a next-generation version of Vim. And before Vim, no company was willing to make a next-generation version of Vi.

Look at Linux.

True innovation often comes from the non-business side of things, where true passion can be found.

3 Likes

Hello @trusktr. This project looks really interesting. I admire people who attempt something this big with quite a few unknowns.

When do you expect to have a functioning prototype?

There’s lots of work left still. I finished porting the minimal requirements for, Matrix4, Vector2, Math, Euler, and Box3 just over a week ago.

I’m planning to do more this week and the weekend.

I imagine it may be months (if not more than a year, but hopefully not that long!) before a working prototype due to the fact that I’m working on it on my free time outside of regular work.

If anyone is interested in helping, that would make a working prototype become reality faster! :slight_smile: If interested, let me know at https://gitter.im/infamous/glas and I can share a document that outlines the progress and TODOs.