Hi, I’ve recently read through the GitHub PR on the topic, shared from the official MaterialX Slack channel. I’d like to offer my thoughts here as I have experience imlpementing MaterialX support into a production render engine. For the record, I am not directly affiliated with the MaterialX project, though I have made some contributions and gave a talk about my implementation in it’s Virtual Touwnhall even last year.
Also for the record, I am not really a user/developer of THREE.js myself, though I have learned about it and used it a bit in a course a few years back. I am a graphics programmer by trade now, and have written rendering engines of many kinds from scratch for both CPUs with pure software rendering as well as for GPUs with/without graphics APIs, in C, C++ and TS, using CUDA, Vulkan, OpenGL and WebGL.
With that out of the way, here’s my condensed thoughts about what THREE.js should consider:
I’ve implemeted over 130 MaterialX nodes as native node for an existing node-based shading system. That way MaterialX native node graphs can also be authored directly, not just imported.
I have also implemented a MaterialX importer that imports from either an .mtlx file or an in-memory MaterialX document (and in the latter case the same importer works when importing from USD by first converting the USD representation into a MaterialX document).
This is heavlly using the graph querying and traversal facilities of the MaterialX core C++ library.
Because the node-based shading system I’ve implemented this for, already does shader code generation, I have used that. So I have not used any of the shader generation facilities of the MaterialX C++ library itself.
While I don’t really see TSL as fully equivalent to a node-based system, it seems enough of one for me to suggest to do a similar thing. Basically, make TSL nodes representing MaterialX nodes, and then make the 2 GLSL/WGSL back-ends of TSL able to emmit appropriate code per-backend for each MaterialX node.
That would have many benefits:
- Users writing TSL can readilly start using MaterialX in their TSL code, freely intermixed with other existing TSL nodes.
- Node based shading graph tools that already emmit TSL could just support emitting the TSL code that use these new TSL nodes.
- The shader generation / transpillation process of TSL → GLSL/WGSL already exists, so the emitted code should work within the context of TSL/THREE.js.
- The footprint of the WASM version of the MaterialX C++ library can be massively reduced, as you can limit it to just the core part of the library, only for importing.
- There should be less friction for shader code generation if it doesn’t even need to deal with the WASM MaterialX library at all for that.
- The whole importing story would get dramatically streamlined, as you’d have direct equivalents in TSL to generate using, mapping from MaterialX graphs, nodes and inputs.
The cost of that is the need to go node-by-node and set-up code generation for the 2 backends.
But you can grab code from the MaterialX library from the code-generation source, in both langauges, to get a head start.
You can then adapt it to how THREE.js uses shader code in each of these backends.
You can then also set up a way to use that code more directly, for users that want to write GLSL/WGSL code more directly, bypassing TSL.
For importing, if you want to use the WASM version of MaterialX C++ library, and want to do the actual importing in JS/TS, maybe you’d want to have a look at other JS projects that do a similar thing, and maybe share some of their code. I haven’t done anything like that, but I would guess that you shouldn’t expect to simply be able to call C++ functions directly from JS without some interop friction.
Alternatively, you could just implement your own DOM traversal in JS, as shown in the GitHub PR.
That would have it’s own set of trade-offs though:
On the one hand you don’t need to bother with the MaterialX C++ library nor WASM at all.
On the other hand, all you get is DOM traversal, so you’d likely end up wrapping that with your own version of what the MaterialX C++ core library is layering on-top of XML traversal in C++ to express concepts of nodes with inputs and outputs connected in a DAG. That might be a whole chunk of work that you’d also need to then maintain, so maybe I wouldn’t recommend that, at least not initially.
As for which nodes to support, I’d recomment for an initial support story to stick with the standard library “pattern” nodes. That is, only nodes that end up participating in node graphs that are upstream from surface-shader inputs. That way, you can also have these nodes used within your own existing (non-MaterialX) materials to drive their inputs as well. You also avoid anything to do with geometry modification like displacement, or emission, or volumetric shading.
You shoul also avoid all PBR / BxDF nodes, like ones used to “construct” custom surface-shaders, like BRDFs and also EDFs/VDFs.
You can then initially support a sub-set of inputs of surface-shader nodes of MaterialX, and just map those to equivalent inputs of your existing materials when importing.
Later, you could look into implementing your own versions of Standard Surface and/or OpenPBR, and I could also provide some tips/insights for that (I’ve also implemented full support for OpenPBR latest spec. version 1.1 so am intimately familiar with the ins and outs of that).
For pattern nodes, you should expect at the very least around 120 node categories.
MaterialX node definitions specify type-variants within each node category.
For example, the Add node has many type-variants for adding integers vs. floats vs. vector2/3/4 vs. color3/4 etc. Many of these nodes also have type-variants for secondary inputs, that can be either of the same value-type as the main input, or a scalar float (which typically gets broadcases when operated against mult-channel types).
MaterialX considers each of these variants to be their own distinct node types.
If you go with that route, you’d end up with north of 700+ nodes, so I would not recomment that.
What almost everyone does instead (and what I have done) is “coalesce” these as much as possible to a single node per category, and have it be configurable somehow by usage code to “explicitly” specify which type-variant is needed, when used.
So, for example if the use needs to scale a 3D vector by a scalar, you want to do something like:
mxMultiply(myVec3, myFloat, tv.Vector3F)
as opposed to something like:
mxMultiplyVector3F(myVec3, myF)
Otherwise it gets annoying, and in JS you could even have it able to be deducted from the arguments in many cases, so that the selected type-variant would not be needed (i.e: TSL very likely already does something like that now, with things like sin() being able to be used for both scalars and multi-component types, without providing additional information).
I’m not expecting to participate in this effort, but just to share my thoughts and experience - MaterialX has its fair share of gotchas and certain aspects aren’t always very clearn, well documented or fully correct.
Best of luck in this 
