What's the purpose of `Fn()` in TSL?

In the TSL docs, there’s no mention of .Fn.

Here is an example that works, using Fn(): https://codepen.io/trusktr/pen/VYjLXBX

Here’s the same example without Fn(), not working: https://codepen.io/trusktr/pen/MYewGLQ

It has this error:

Uncaught TypeError: Cannot read properties of null (reading 'If')
    at If (three.webgpu.js:4264:42)

and that line looks like this:

const If = ( ...params ) => currentStack.If( ...params );

Here’s one with no function at all, also not working, with the same error (makes sense it has the same error): https://codepen.io/trusktr/pen/emzNroK

What’s Fn for exactly?

See here:
https://github.com/mrdoob/three.js/wiki/Three.js-Shading-Language#function

Generally, it describes a function-ish entity. If the parameters are only TSL values and are well defined, a Fn is compiled into a GLSL/WGSL function. Otherwise it is inlined and it does not exist as a standalone function. However, IMHO, one of the hidden advantages of Fn is that it can mix JS and TSL statements (e.g. if and If or + and add) - so you can construct conditional functions that have different body commands - I use this heavily for the Disfigure library.

At a low-level perspective, Fn only defines a subtree in the AST (abstract syntax tree), which may later become a function or an expression. So it is first processed by JS, compiled, executed, and then the result is processed by the TSL engine that compiles it ito GLSL or WGSL commands.

At a technical level Fn wraps a normal JS function, but does some additional processing, which is needed for TSL. If you use the function without wrapping it in Fn, these additional things will not be done, and some other things will fail.

2 Likes

Thanks. Looks like the documentation is sort of in transition between old docs (written in HTML files), new docs (written in JSDoc comments) (EDIT: oh these are generated to HTML side by side with hand-written docs?), and the wiki.

I read the page you linked, but some things aren’t immediately obvious. For example, you mentioned

and the wiki page mentions

The main difference is that Fn() creates a controllable environment, allowing the use of stack where you can use assign and conditional , while the classic function only allows inline approaches.

but then it does not go and show examples comparing the mentioned differences. The wiki page proceeds to show an example doesn’t demonstrate assign or conditional stuff, but a simpler function that works either way with or without Fn:

// tsl function
const oscSine = Fn( ( [ t = time ] ) => {

	return t.add( 0.75 ).mul( Math.PI * 2 ).sin().mul( 0.5 ).add( 0.5 );

} );

// inline function (no Fn)
export const oscSine = ( t = time ) => t.add( 0.75 ).mul( Math.PI * 2 ).sin().mul( 0.5 ).add( 0.5 );

Maybe this can be updated to precisely explain the differences.


As one example, this code does not throw or log any error, and does not work:

const result = someNode.mul(otherNode)

result.lessThanEqual(0.5).discard() // does not work

mat.opacityNode = result

codepen

This code also does not work, but it does throw an error:

const result = someNode.mul(otherNode)

If(result.lessThanEqual(0.5), () => Discard()) // does not work, throws "Cannot read properties of null (reading 'If')"

mat.opacityNode = result

codepen

But both of the following work (the 1337 conditional check successfully logs the WGSL code to console):

const result = someNode.mul(otherNode)

const opacityNode = Fn(() => {
  result.lessThanEqual(0.5).discard()
  return result
})()

mat.opacityNode = opacityNode

codepen

const result = someNode.mul(otherNode)

const opacityNode = Fn(() => {
  If(result.lessThanEqual(0.5), () => Discard())
  return result
})()

mat.opacityNode = opacityNode

codepen

It isn’t clear why this is required. Seems to me like both versions without Fn (chaining vs If), could (should?) just work, as neither of those graphs (both attached to opacityNode) rely on any arguments such as material being passed in from the renderer, so they seem fully standalone.

Another question may be, what is it about the “node graph” (that is not very exposed via TSL, and remains more of an internal detail), that would prevent those things from simply working?

Is it that branches that lead to nowhere (i.e. dangling paths not leading to the final output node) are not tracked? If so, could a simple no-op merge fix that to join dangling paths into the final result? For example:

const result = someNode.mul(otherNode)
const discardCheck = result.lessThanEqual(0.5).discard()
mat.opacityNode = result.merge(discardCheck) // either the first node is only considered for the output
// or
mat.opacityNode = discardCheck.merge(result) // or the last node is only considered for the output

and

const result = someNode.mul(otherNode)
const discardCheck = If(result.lessThanEqual(0.5), () => Discard())
mat.opacityNode = result.merge(discardCheck) // either the first node is only considered for the output
// or
mat.opacityNode = discardCheck.merge(result) // or the last node is only considered for the output

which implies that the arrow function for Discard() would not be needed, because Fn would no longer be needed, and Discard() would simply be a declarative node connected in the graph:

const discardCheck = If(result.lessThanEqual(0.5), Discard()) // merges two nodes into a boolean.

The idea is, if the graph of nodes are ultimately connected, then they should all just work. And if dangling paths are too complex or undesirable to handle (too much perf overhead to traverse and find them?) then the merge idea could solve that: there’d always be an easy-to-follow path from any top-level input node to the final output node, or vice versa.

Can those use cases be made to just work without Fn?

It looks like you are aiming at complete and deep understanding of Fn and other TSL things. As far as I know, there is no such complete guide or tutorial. And my hope is than when you reach this level of understanding, you could write the tutorial.

My approach is more practical - instead of first building a solid theoretical knowledge and then writing code, I experiment with just the things I need. This allows me to use TSL even if my knowledge is rather scattered and disorganized.

Here is example of conditional code - this is JavaScript’s if, not the TSL’s If. The result is that in this way the body of a TSL function is changed before TSL processes it. So, it is possible to optimize, fine-tune, or customize code in the shader, instead of putting all options inside the shader code and branching there.

// a fake function, just for illustrative purpose
var myfunc = Fn( ({x,y,smooth})=>{

	var result = float(5).toVar();

	// note the use of javascript `if`
	if( smooth.value ) {
		var k = x.pow(y).abs().mul(100).sin().toVar();
		result.addAssign( x.smoothstep(k,y.sub(k)) )
	} else {
		result.addAssign( x );
	}
	
	return result;
})

When used with myfunc(1,2,true), the compiled code is:

nodeVar0 = 5.0;
nodeVar1 = sin( ( abs( pow( 1.0, 2.0 ) ) * 100.0 ) );
nodeVar0 = ( nodeVar0 + smoothstep( nodeVar1, ( 2.0 - nodeVar1 ), 1.0 ) );

When used with myfunc(1,2,false), the compiled code is:

nodeVar0 = 5.0;
nodeVar0 = ( nodeVar0 + 1.0 );

Note, that only one of the branches exists in the shader.


Now, if the TSL If is used:

If( smooth, ()=> {
	var k = x.pow(y).abs().mul(100).sin().toVar();
	result.addAssign( x.smoothstep(k,y.sub(k)) )
}).Else( ()=> {
	result.addAssign( x );
});

the compiled code has both branches:

if ( nodeVar3 ) {
	nodeVar4 = sin( ( abs( pow( 1.0, 2.0 ) ) * 100.0 ) );
	nodeVar0 = ( nodeVar0 + smoothstep( nodeVar4, ( 2.0 - nodeVar4 ), 1.0 ) );
} else {
	nodeVar0 = ( nodeVar0 + 1.0 );
}

In some awkward sense, we may use JavaScript as preprocessor to TSL :open_mouth:

3 Likes

Might be worth clarifying what the stack means in this case. Making a call to that If node in some random place in code would probably be pretty meaningless.

But because it is happening “within” that Fn call, it has a context and can generate that block of code.

You need to create that context. When TSL encounters the Fn with this callback, it can do magic. So if there are any side effects they can be applied in the right place (since you don’t return a discard for example to connect to other nodes).

Notice in @PavelBoytchev examples what is being returned to whom. Nodes can be chained, but you always get a reference (var foo) that you further pass into other nodes.

If i imagine is not returned to anyone, so the side effects would matter.

2 Likes

Yeah, that’s what I figured, that it’s for tracking the side effects.

But I don’t get why we need closures for that yet. In other words, I imagine an alternative graph where if something is connected to the graph, then it gets included (doesn’t need a function invocation to track it).

So, imagining that possibility, I would guess that maybe the reason is that traversing the whole graph to find all those dangling side effects would be too complex? And having the context with function invocation is some way to simplify that to make it explicit?

Basically I’m just imagining a pure node graph, for example:

      SomeValue
         |
      Boolean // SomeValue connected to Boolean would coerce it to true/false or 1/0
         /\
        /  \
       /    \
    IfTrue IfFalse
      |       |
   Discard SomethingElse
                |
             Output // the Output is still the final result, regardless if Discard was a side effect i.e. a dangling branch

In this paradigm I’m describing, there’s no need to know about a concept of a “context” and requiring special function closure invocations to achieve results, the only thing required is to simply connect nodes together.

After all TSL is “nodes” right? But I think that TSL has gone beyond the scope of nodes (and it’s perhaps a good thing when it comes to shader-like code in JS).

Part of why I’m wondering about this is, because I’m trying to use the Node classes directly in that manner (in theory by simply connecting all needed nodes together, without any need for special function invocations),

but I’m not having luck reproducing that simple example in that format yet.

(If anyone knows how to complete the attempted example there, I’d be curious to see).

The reason to want to do that is simply: if we have an outcome simply representable with a node graph (just a web of connected nodes, no other constructs), then we can easily represent it with a visual GUI in a one-to-one mapping (all node map directly to GUI boxes, all node connections map directly to GUI lines). The logic node graph could very well be the main data structure for the UI.

And that makes me wonder if some TSL features are too abstract for (transcended beyond) a node graph concept to easily model with a graph UI (for example, I do not see If or Fn nodes in the nodes Playground, and my guess is that’s because If, Fn, and similar TSL constructs have transcended beyond the node paradigm into something of their own that isn’t immediately simple map to node representation (even if those things technically do extend from Node)).

There is a very rudimentary reason for using TSL Fn instead of JavaScript function.

Javascript has values like floating point values, functional values, boolean values etc. To build a node, you must wrap this value in a node. For example, how to make 2.34 into a node to be used by TSL? You write float(2.34), so float(...) is a function that converts a number into a node containing that number.

How to make a function f into a node? You use Fn(f), so Fn(...) is a function that converts a JavaScript function into a node containing that function.

Apparently one of the benefits of Fn is that it can also provide type information (via its layout parameter or setLayout method). Without such information a shader function cannot be written, because shading languages are very strict-typed, whereas JavaScript is practically typeless.

1 Like

You may be right in the perspective and benefit of TSL code, but it seems to escape out of the node paradigm (so it’s not really “nodes” anymore).

I want to be able to express any TSL as an equivalent node graph, and right now Fn seems to prevent me from doing that, or at least it is not obvious how to do it.

As an example, in Blender nodes, conditional logic is fully expressible via the node graph. See this video, where the Switch node at 4:35 allows expressing conditional logic, and there is no notion of function closures needed to achieve the conditional:

Blender users that use only the GUI can express everything purely via node graph.

So what I’m saying is, TSL seems to be doing something that, when looking at the actual node system, is not very node-based. What I want is to be able to fully represent anything I want to do with nodes only. F.e. new ThisNode, new ThatNode, etc, and connect them all together, without having to make function closures and executing them (because that doesn’t exist in a paradigm like a node GUI where the only thing possible is to connect nodes together).

I want to be able to fully express what I want purely as nodes because that means I can then very easily make a new node GUI for it (which is something I want to try).

@dubois do you get what I mean? I know you have some experience making a node GUI. Were you able to implement the equivalents of what we can do with Fn in TSL, but purely with node GUI (i.e. no function closures)?

Im not sure if I understand. What might be throwing you off is that if essentially has multiple outputs, while something like add does not.

Likewise a custom function could have multiple inputs and multiple outputs.

Edit

Yeah a switch doesn’t have, it’s a ternary operator. I haven’t actually implemented discard in my gui. I’d have to look at it.

any ideas how to complete the conversion of the example in How to use the `Node` classes instead of TSL? - #2 by trusktr to pure node instances without TSL?

I think I’m at the edge of understanding what you need to know about Fn() and at the edge of my own understanding of Fn(). The situation looks to me like this:

A person orders a cake. The waiter brings the cake and says it can be eaten with a fork or with a spoon. The persons says, I do not want to eat it with a spoon. Please, I want a cake that can be eaten with a fork. Bring me another cake, that can be eaten with a fork.

Now seriously. Traditional node systems are usually restricted when you need to express something complex. They are perfect for expressions, but they struggle with procedural code (i.e. a sequence of commands). Blender’s node are for building expressions. If you need real functions, e.g. you need a node that acts as function, you have to use … closures (more info).

Now, in TSL if you can express the logic with expression, you do not need Fn(). Here is a demo where dynamic coloring is done without Fn():

https://codepen.io/boytchev/full/yyJavey
image

It might be somewhat fair to consider that Fn() creates a node holding the definition of a user-defined custom function. You may check the source code of Fn() in TSLCore.js to see it actually uses ShaderNode under the hood but does some additional (and unavoidable) processing of inputs.

So, if you want to convert some old procedural shader to nodes and avoid functions, convert it to expression and then to nodes.

As for the problem that some internal definitions are not exported, I have no idea whether they are not exported because it is considered they would not be useful for outside, or there is some specific reason why they should not be exported. Maybe it might be good to file a request at the GitHub Three.js repo asking for export of all internal nodes and some argumentation or usecases to show the benefit of having these nodes accessible.

1 Like

TSL is amazing, but the syntax—especially the shader syntax—is terrible.
It becomes completely unreadable when multiple mathematical operations are involved. I really hope this isn’t the final state, because otherwise I don’t see a good future for the custom shader workflow.

Shaders are already hard enough as they are; adding a verbose, non-mathematical syntax on top of that makes them almost impossible to work with.

Just my five cents :slight_smile:

1 Like

That’s interesting! I hadn’t seen that before but I’m not quite a Blender expert yet.

This isn’t quite like the cake metaphor though:

I was indeed aware of using expressions without Fn, but when trying to add a conditional Discard expression, that’s when I realized it doesn’t work without a closure/Fn.

For example, forking your pen, I added this:

// This doesn't work
r.greaterThan(0).discard() // this is a dangling node branch
const colorNode = select(flip, vec3(r,g,b), level.oneMinus())

// but this does
// const colorNode = Fn(() => {
// 	r.greaterThan(0).discard()
// 	return select(flip, vec3(r,g,b), level.oneMinus())
// })()

To me, it isn’t intuitive why a closure is needed for that case: I am not closing over any variables (the purpose of a closure), not accepting any function arguments, and not using any context (such as material global variables). It’s only a Discard statement based on a conditional for a fully static expression.

It might just be that this area needs some work.

It’s a dangling branch in the graph, which is definitely expressible by connecting simple boxes with lines (f.e. what I tried to write in that pen). I haven’t tried a dangling branch like this in Blender yet though, and I wonder how it’ll do.

For some reason I haven’t understood yet, TSL can’t handle this type of expression yet. My initial guess (total guess) is dangling nodes don’t flow between start/finish, so don’t get seen by the build algo. Hoping to find out sometime!

Indeed, I got a response here that they should be exported, so it’s just a small oversight.

1 Like

I love it. Being able to patch the API like this for example is great. (I do hope that those shadow nodes APIs can be made into more ergonomic extension properties (like material.positionNode) instead of patching, but hey it worked).

The new shader parts are organized in relevant classes rather than a separate string lib, so patching/overriding/extending logic of a method feels right at home. Plus it’s more readable than string patching.

Yeah math expressions can be more verbose, but I think it’s a good tradeoff, especially because no build needed (very aligned with web standards), IDE support (TypeScript) out of the box (though it still needs a lot of improvements, WIP), and backend agnostic (GLSL/WGSL, maybe others outside of web at some point like HSL/MSL/Slang/etc).

The math part is really bad if you are serious about shaders. A complex shader quickly becomes unreadable when everything is expressed with sub, mul, and long chained calls.

GLSL is math — it must look like math.

Just compare the code below. This is only a very simple example, and if this issue is not addressed, shaders will be close to impossible to work with, in my opinion. The TSL expression is mathematically unreadable. Normal algebraic operations must be allowed instead of these verbose, text-based expressions. As it stands, it’s honestly horrible, and I have to be truthful about how I feel.

Now imagine a full shader written this way… yikes, and this is a very basic example, a full GLSL shader is not readable anymore, not to mention that all my shaders that Ihave written in GLSL now Ihave to translate to TSL. Sure, I can do it with AI, but it will be unreadable to me.

GLSL

f *= f * (3. - 2. * f);

TSL

f = f.mul( f.mul( float(3).sub( float(2).mul(f) ) ) );

I think it would be normal that this does not work (unless TSL plays too smart). I see at least a few problems to make this run:

  • there are expressions (things that return values) and statements (things that act, without returning value). The code above is expression, so it cannot contain statements, because discard is a statement in GLSL. The same problem happens even if JS, just try var b = a>0 ? return 4 : a++.
  • in GLSL discard is a statement, a part of the syntax construct, like return, if and so on. It is not a function, it does not return a value, in cannot be chained in an expression.
  • in WGSL there is no discard, it uses just return and to handle return you need to run it inside a function. As long as TSL wants to be GLSL and WGSL friendly, this might be another reason for required Fn()
  • when TSL compiles code, it removes part of the node tree that does not result been connected to the root. It might be quite possible that r.greaterThan(0).discard() would not connect anything to the node tree – see, greaterThan just returns a boolean, it is not if or select, it is not a control function. Nothing checks this value and react accordingly. It is just as this code r.sin().discard().
  • it might be possible that TSL creates disgard or return statement whenever it meets discard(), but it is hard to guess where to put it in the final code. If you have statements, they are ordered sequentially. But in expressions, you can never guess where it will be embedded, often an expression could end up in several places (because of inlining) so you may get multiple discard’s scattered all around the code.

All these are hypotheses, I’m not engaged with the development of TSL, nor the node system, but I had experience in making compilers and interpreters some 30 years ago.

I do not see a problem for introducing a special node that represents custom function in your node UI. But I see a problem fitting JS in it, unless you welcome the struggle to parse JS code.

Currently TSL piggybacks on JS and uses its parser, syntax checking, evaluations and so on. Maybe a standalone parser of TSL source code (not JS code as it is now) could resolve your issues. As well as it could revert to the dominant infix notation for expressions.