Porting Three.js to WebAssembly with AssemblyScript

Hello!

We are porting Three.js to AssemblyScript (a strict subset of TypeScript that compiles to WebAssembly).

:point_right: github.com/infamous/glas

The project is in the early stages, but we already have a number of classes ported over along with passing unit tests. For example here is Matrix3 and its unit tests.

Join us over in the repo or chat with us at gitter.im/infamous/glas.

6 Likes

I love your enthousiasm.
But, and this is a legitimate question, I really donā€™t know and Iam not implying anything, is it really as simple as just converting code?
Do you have access to the canvas context? To the WebGL APIs?

1 Like

For most of the code, yes, it is as simple as converting code (including the unit tests), as you can see in the Matrix3 example I linked above.

At the moment, no: the WebAssembly context does not have direct access to the canvas. Once we get to porting WebGLRenderer, then instead of having a canvas reference inside of it, we will have a stack of render commands (a command queue) in the WASM context. With some glue code, weā€™ll send the stack of commands over to the JS side, and on the JS side there will be a handler that receives the commands (which contain shaders, attributes, etc) that will pass them directly to the GPU.

This will be the hardest part of the porting effort.

Similar things have already been done for Canvas 2D APIs. For example, @jtenner, an active contributor to AssemblyScript, and creator of the as-pect unit testing framework for AssemblyScript which weā€™re using for the unit tests in glas, created a project called as2d in which he demonstrated how to send commands from the WASM side to a canvas 2d context on the JS side. Now weā€™ll use a similar technique to render to a canvas webgl context.

Itā€™s a process, and it will take some time to implement for sure.

Of course, the WebAssembly GC spec is underway, and at some point in the near future we will be able to reference DOM and JS objects from WebAssembly. The WebAssembly Reference Types spec is already released behind a flag in Google Chrome (I havenā€™t checked other browsers on this yet), which is a pre-requisite for WebAssembly GC, and finally DOM/JS interop (including importing JS modules into WASM, and vice versa).

Itā€™s the future!

We can use all the help that we can get, if youā€™re interested!

4 Likes

Do you have any idea of performance of AssemblyScript vs C/C++/Rust -> Wasm?

I had a look but couldnā€™t find anything related to this.

My thinking on using Wasm with three.js has been, rather than converting everything to Wasm, we should figure out the hot paths and rewrite them in C/C++/Rust -> Wasm (or whatever language).

The maths library would be an obvious candidate for this. There may already be highly optimised math libraries that we could repurpose here.

Rewriting non-hot paths in wasm may give less of a performance boost than you expect. For example, if something takes 50ms in JS, vs 2ms in Wasm, but only runs a couple of times, then you are not gaining anything a user will notice and your effort is essentially wasted. As a result, I would concentrate on hot paths at first.

EDIT: I found this page (summarised here) comparing C/Rust/AssemblyScript and also giving a nice summary of replacing a hot path in squoosh.app with Wasm.

Surprisingly, AssemblyScript comes out very strong - nearly as performant as C, and a very small file size.That said, they were testing a single file so I would not consider this conclusive.

I still think that it would be worth adapting a pre-built and battle tested game math library written in C++ though. Simple converting JS->AssemblyScript->Wasm is unlikely to give the best possible performance.

1 Like

Here I played with AssemblyScript little bit for heavy calculations in loop:

And I realized that we are facing with different aspects when we want good performance right now.

First problem: that Wasm is sandboxed for security reasons and Wasm<->JS interoperation is pretty heavy. You need to calculate all stuff in one big wasm block in order to avoid it. WebAssembly not an assembler, and you canā€™t rewrite one small piece of code with it to get performance boost. You can make performance worse accidentally if you call wasm too much.

Second problem: sometimes you have unexpected results for different data structures in AssemblyScript (and probably in rust/c++, I am not try). For example when I return a class from my function I accidentally get object explosion problem and x10 performance drops that become worse than JS.

Also If you want to pass big array to/from wasm you should know best way how to do it.

Also recently TensorFlow created new wasm backend and they get x10 boost for float32 calculations on CPU.

All this is good but you should know! That probably on your machine In most of examples you have a bottleneck on GPU, not on CPU. At least I get such results. You can try to measure CPU/GPU load by yourself with this tool. For future we should think about WebGPU too, since we have bottleneck on a video-card.

That might the case for most mobile devices, but at least on my laptop itā€™s the opposite.

This is a very good point, but as far as Iā€™m aware itā€™s something that the wasm team are working to reduce and should be less of an issue in the future.

This article of the wasm roadmap explains it well. Itā€™s pretty big so skip to the Small modules interoperating with JavaScript section:

@trusktr the more I research on this, the more it seems to me like converting big libraries completely to wasm is not something that anybody is doing right now. And that includes huge companies that certainly have the resources to do so. Instead, they are converting single big tasks that can easily happen off the main thread and take a long time in JS, like image recognition.

Your project is certainly a valid project for experimentation and learning, but Iā€™m not convinced the performance gain will be worth it. Otherwise, why arenā€™t other people doing it?

Of course, perhaps they are and I just havenā€™t heard about it!

3 Likes

Interesting

They have an example with parser, itā€™s pretty big block of code. I am talking about ~15 instructions in wasm module that you can call from js 3000 times in one frame. In another words: all loops need to be inside wasm sandbox, except requestAnimationFrame loop.

Here is a demo with FPS counter: Wave Equation (as you saw in the other article you found, AssemblyScript is comparable to Clang and Emscripten).

AssemblyScript is using WebAssembly/binaryenā€™s optimization pipeline (by Alan Zakai, same person who made Emscripten).

That may be one way to do it, but Iā€™m aiming for a great end-developer experience (everything being in a single easy-to-use language like AssemblyScript).

Bullet (Ammo) in Three.js is a black box, and developers can not simply fork it and update things they need (for sake of example imagine someone needs to customize something in Bullet).

With a simple system in place for all code (f.e. AssemblyScript), the end user can easily fork any part of the code, and even be more likely to make contributions back to the project.

Replacing hot paths might be ā€œenoughā€, but I donā€™t think it will lead to the developer experience that Iā€™m imagining.

AssemblyScript now supports threads. The WebAssembly GC spec will give us the ability to reference ā€œhost objectsā€ such as DOM and JS objects inside WASM is underway (this includes WASM modules directly importing ES Modules, and vice versa). With these advancements weā€™ll be able to make optimizations by avoiding the current cost of WASM-JS communication needed to access a canvas, and by placing different parts of the system on separate threads (:drooling_face: :drooling_face: :drooling_face:), all while maintaining a developer experience that web developers are familiar with.

AssemblyScript is simply easy to use and it works great with existing TypeScript tooling (VS Code Intellisense, Prettier, ESLint, Webpack, etc) as long as we donā€™t use the few features that are AS-specific, which is what weā€™re doing in glas.

This is a possibility. If we did this, Iā€™d want to port the tool over to AS too, with the goal of having everything in the single-language and with the best end-dev experience.

This is true, but in Firefox theyā€™ve already made huge improvements:

Calls between JavaScript and WebAssembly are finally fast :tada:

The WebAssembly GC features I linked above will allow us to totally bypass this problem when that comes out.

Yep! So the design in glas will be that all code runs in WASM (all code is AssemblyScript), and for every frame the WASM module will send the JS a single command queue with a single WASM-JS call, to minimize this issue for the time being.

If we were to replace only various hot paths with WASM, this would mean weā€™d need to call into WASM multiple times.

Interesting! Thanks for pointing that out. In glas weā€™re using only top-level classes and keeping it simple like Three.js (weā€™re not changing the structure except as needed to make it work in AS).

Nice! Thanks! Thatā€™ll come in handy once we get to WebGLRenderer.

This is true! It probably depends on the specific application (f.e. @looeeeā€™s scene is more CPU heavy on his laptop, and use of a lot of physics would increase CPU, etc).

At work, one of the bottlenecks we have is rendering many of the same thing, and each individual mesh causes one expensive GPU call.

I have some plans with InstancedMesh to reduce the number of GPU calls by automatically instancing meshes in a subtree, so that we can use the scene graph tree the way we do now (organizing transforms in a hierarchy) instead of having to manage separate lists of instanced objects (this is spread across two issues, sorry):

Iā€™ve imagined how to do it, I just need to implement it.

You can try to measure CPU/GPU load by yourself with this tool.

Niiiiiiiice! Thanks! Thatā€™s neat!


On a different note, WebAssembly can be compiled to native assembly which means that eventually we can have an OpenGL ES target, not just a WebGL target. Here are some tools to do that:


I guess Iā€™m ambitious. :smiley:

Plus, if they already have big applications written in JS, and they have business needs, they perhaps donā€™t want to afford the cost.

But in my case, Iā€™m doing it for free, becauseā€¦ hack yeah :computer::floppy_disk::zap:!

Because, it takes time and effort. Most people want to use what exists to make products. Most businesses use what exists so they can make money, not so they can invent a free tool that no one will pay them for. This is why, despite that WebGL is now in every browser, big companies still use 2D HTML for UIs, and no big companies write WebGL UI because they donā€™t need to, and game companies use existing native tech for games, because they donā€™t need to port to WebGL (and some tools compile to WebGL). When you look at business needs, this all makes sense.

When people take passion, and put business needs aside, then the best things come forward; things like Three.js, AssemblyScript, NeoVim text editor, and many other free projects that business-oriented organizations are less likely to create.

And then, once those amazing free projects are released into the wild, suddenly businesses realize they can use these tools for their own profit. Many companies started using Three.js, for example, but no companies wrote their own engine because, they didnā€™t need to. Theyā€™ve started using AssemblyScript, because they see its potential after its creation.

Well, companies donā€™t start using NeoVim, thatā€™ll always be from passion for passion. But Before NeoVim, no company was willing to make a next-generation version of Vim. And before Vim, no company was willing to make a next-generation version of Vi.

Look at Linux.

True innovation often comes from the non-business side of things, where true passion can be found.

3 Likes

Hello @trusktr. This project looks really interesting. I admire people who attempt something this big with quite a few unknowns.

When do you expect to have a functioning prototype?

Thereā€™s lots of work left still. I finished porting the minimal requirements for, Matrix4, Vector2, Math, Euler, and Box3 just over a week ago.

Iā€™m planning to do more this week and the weekend.

I imagine it may be months (if not more than a year, but hopefully not that long!) before a working prototype due to the fact that Iā€™m working on it on my free time outside of regular work.

If anyone is interested in helping, that would make a working prototype become reality faster! :slight_smile: If interested, let me know at infamous/glas - Gitter and I can share a document that outlines the progress and TODOs.

1 Like

@trusktr you can use new rollup plugin for AssemblyScript.

1 Like

@munrocket Thatā€™s neat. My goal at the moment is that the user of glas will also write AssemblyScript code to write the WebGL application. But that plugin could make it easier for Rollup users to then import their WebGL app and finally run it. Thereā€™s also a Webpack loader.

Once we can render something to the screen, then making it easy to run the app will be the next step.

My goal at the moment is that the user of glas will also write AssemblyScript code to write the WebGL application.

Hopefully this doesnā€™t come across too bluntly and I say this as someone whoā€™s excited to see where web assembly goes and how it can be integrated into my work but this would kill my interest as a potential user. It would mean I canā€™t try it in a project without going all in and forces a new language on me.

Where I see the most value in web assembly is converting the hot paths or WASM-friendly code of an application like rendering work, mesh processing, or matrix math to something like webassembly but otherwise the lighter application glue, logic, scene setup, etc can remain in Javascript. Otherwise it just doesnā€™t seem sustainable. There will be frameworks in Rust for WASM that require you to use Rust, C++ that require C++, and AssemblyScript that require AssemblyScript. How will they ever be able to play nice with each other? There will be tools and libraries built to only work in the specific language. I just see it further fragmenting the web if this is the general approach taken and then I donā€™t think anyone wins.

Maybe itā€™s too early for that, though. I know thereā€™s a lot of complexity to it with JS interop. Especially with the async compile requirements and (maybe?) remaining overhead of wasm calls but Iā€™m hopeful itā€™ll get there at some point. Iā€™ll be keeping on eye on this if only to see how performance turns out either way!

1 Like

For now, the initial goal of the project is that everything is 100% AssemblyScript (which is TypeScript (which is Javacript (so the code isnā€™t very different from what you already write (thatā€™s why we chose AssemblyScript)))).

It has been super easy to start writing AssemblyScript: the intellisense in VS Code works, itā€™s just TypeScript. It is super easy to compile. Thereā€™s hardly any difference from writing TypeScript with a single tsc command to compile to JavaScript compared to writing AssemblyScript with a single asc command to compile to Wasm, and 20 lines of JS code to launch the application. To make thing easy, the final working example will have a JS API that is simple to use to ā€œlaunchā€ the Wasm application and for everything to just work.

The goal of glas is a 100% AssemblyScript port. Replacing only the hot paths of Three.js is another way to go, but that would be a different project altogether, and someone could totally take on that project (and maybe even re-use parts of the glas project for that by importing specific pieces); and that sort of project could potentially even be merged into Three.js, while merging a 100% AssemblyScript port is probably not likely.

In the future, Wasm will support ES Modules, and we will be able to import JavaScript (TypeScript) into Wasm (AssemblyScript), and vice versa. This will be totally rad, because as web developer weā€™ll be using what we already know (the JavaScript language (with types), as opposed to Rust or C++).

In the future Wasm (AssemblyScript) will also be able to reference DOM objects.

At that point, we could be writing AssemblyScript just as if it is TypeScript (JavaScript), and manipulate the DOM (f.e. get a canvas element, make a webgl context, pass it into the renderer, etc), while everything runs as WebAssembly instead of JavaScript. This will be a seamless environment both for the end user, as well as for developers. It will create a better ecosystem of collaboration where an end user isnā€™t afraid to propose a modification because the implementation code is written in the same system as the user code.

As an example, if Ammo.js physics (which ships as part of Three.js) has an issue, how do you propose a change? You need to go find the Bullet source code which is C++, propose a change there, hope it merges then finally bring that change into Three.js. Thatā€™s not ideal.

By having a system that is entirely in one language, everyone can benefit (it would be neat to port Bullet to AssemblyScript too, just so it isnā€™t some foreign black-box that no one in the community can understand).

Thatā€™s the ideal world that weā€™re aiming for with glas; one system in one language that works with web technology in a familiar way (you already know JavaScript).

One thing that may be possible is that once interface types (JS/DOM access) and ES Module support are out, it may be possible to import bits and pieces of a library like glas and orchestrate the composition in JavaScript.

Lastly, I personally donā€™t want to manage the Three.js source code along with bits and pieces of Wasm code. I prefer to write all in one coherent system.

Thatā€™s what weā€™re curious about too, and this is an experiment to see what happens!