Calculating vertex normals after displacement in the vertex shader

Hey community,
I tought I’d share the solution to calculating smooth shaded normals in the shader since it took some asking around to find the solution. I did not see any three.js implementation of this.

So, if you’re displacing the vertices in the vertex shader (with some noise in my case), you will need to recalculate also the normals in the shader if you want to use lighting.

You can use the derivatives dFdx and dFdy to calculate flat shading in the fragment shader.

However if you want to use smooth shading, it is a bit harder.

The solution came from this old article and a bit from this example and this other example from spite.

Basically, you can calculate the normal by the cross product of the tangent and bitangent.

And you can calculate the tangent and bitangent by calling the displacement function with the neightbour position.

This is what it boils down to:

vec3 tangent = neighbour1 - position;
vec3 bitangent = neighbour2 - position;

vec3 normal = normalize(cross(tangent, bitangent));

If instead you’re using a texture to do the displacement, you can sample the neighbour texel. This is shown here.

I did a minimal codepen example of this:

14 Likes

its-so-shiny_fb_1017995

Are the scrolling seams a result of a scrolling noise texture or an artefact from the calculations :thinking: ?

Not sure if they are always there, but can see some at everything set to 0.5:
Screen Shot 2020-07-11 at 00.20.03

@mjurczyk they’re artifacts caused by the simplex noise function. They disappear if you use sin() as displacement for example.

I don’t know why that happens ¯\_(ツ)_/¯

2 Likes

I’m trying to make this work by default when using material.displacementMap.

Unfortunately the solution presented here only works with a plane.

How to make it so it works with any geometry?

This is what I have so far:

vec2 texelSize = vec2( 1.0 / 512.0, 1.0 / 512.0 ); // temporarily hardcoding texture resolution

float dx = texture2D( displacementMap, vUv + vec2( - texelSize.x, 0 ) ).x - texture2D( displacementMap, vUv + vec2( texelSize.x, 0 ) ).x;
float dy = texture2D( displacementMap, vUv + vec2( 0, - texelSize.y ) ).x - texture2D( displacementMap, vUv + vec2( 0, texelSize.y ) ).x;

vec3 tangent = cross( objectNormal, vec3( 0, dy, 1.0 ) );
vec3 bitangent = cross( objectNormal, vec3( dx, 0, 1.0 ) );

transformedNormal = cross( tangent, bitangent );

Almost…! :sweat_smile:

So seems like this is not possible in the vertex shader without also providing tangents… :thinking:

@mrdoob check out the full code in the codepen I posted, there is a function that uses the following approach to compute a possible tangent :nerd_face:

http://lolengine.net/blog/2013/09/21/picking-orthogonal-vector-combing-coconuts

I’ll paste the relevant code here for reference

// the function which defines the displacement
float displace(vec3 point) {
  return noise(vec3(point.x * frequency, point.z * frequency, time * speed)) * amplitude;
}

// http://lolengine.net/blog/2013/09/21/picking-orthogonal-vector-combing-coconuts
vec3 orthogonal(vec3 v) {
  return normalize(abs(v.x) > abs(v.z) ? vec3(-v.y, v.x, 0.0)
  : vec3(0.0, -v.z, v.y));
}

// ...

vec3 displacedPosition = position + normal * displace(position);

float offset = ${SIZE / RESOLUTION};
vec3 tangent = orthogonal(normal);
vec3 bitangent = normalize(cross(normal, tangent));
vec3 neighbour1 = position + tangent * offset;
vec3 neighbour2 = position + bitangent * offset;
vec3 displacedNeighbour1 = neighbour1 + normal * displace(neighbour1);
vec3 displacedNeighbour2 = neighbour2 + normal * displace(neighbour2);

// https://i.ya-webdesign.com/images/vector-normals-tangent-16.png
vec3 displacedTangent = displacedNeighbour1 - displacedPosition;
vec3 displacedBitangent = displacedNeighbour2 - displacedPosition;

// https://upload.wikimedia.org/wikipedia/commons/d/d2/Right_hand_rule_cross_product.svg
vec3 displacedNormal = normalize(cross(displacedTangent, displacedBitangent));

But I think that code doesn’t work with texture maps?

@mrdoob in your example, have you tried just computing the normal from the slope? (I guess it’s like flat shading in the fragment shader using derivatives)

transformedNormal = normalize( cross( dx, dy ) );

But I think that code doesn’t work with texture maps?

yeah, unfortunately it applies to world space.

In UV space it’s different, because you don’t know the relation between the UV space and the world space. For example, you could try fiddling with the the world space offset and texelSize, but it won’t work for every mesh out there

// the function which defines the displacement
float displace(vec2 vUv) {
  return texture2D(displacementMap, vUv);
}

// http://lolengine.net/blog/2013/09/21/picking-orthogonal-vector-combing-coconuts
vec3 orthogonal(vec3 v) {
  return normalize(abs(v.x) > abs(v.z) ? vec3(-v.y, v.x, 0.0)
  : vec3(0.0, -v.z, v.y));
}

// ...
vec3 displacedPosition = position + normal * displace(vUv);

float texelSize = 1.0 / 512.0; // temporarily hardcoding texture resolution
float offset = 0.01;

vec3 tangent = orthogonal(normal);
vec3 bitangent = normalize(cross(normal, tangent));
vec3 neighbour1 = position + tangent * offset;
vec3 neighbour2 = position + bitangent * offset;
// demo for now, the direction should be the same of the tangent and bitangent
vec2 neighbour1uv = vUv + vec2(-texelSize, 0);
vec2 neighbour2uv = vUv  + vec2(0, -texelSize);
vec3 displacedNeighbour1 = neighbour1 + normal * displace(neighbour1uv);
vec3 displacedNeighbour2 = neighbour2 + normal * displace(neighbour2uv);

// https://i.ya-webdesign.com/images/vector-normals-tangent-16.png
vec3 displacedTangent = displacedNeighbour1 - displacedPosition;
vec3 displacedBitangent = displacedNeighbour2 - displacedPosition;

// https://upload.wikimedia.org/wikipedia/commons/d/d2/Right_hand_rule_cross_product.svg
vec3 displacedNormal = normalize(cross(displacedTangent, displacedBitangent));
1 Like

Hi, This appears to work on the x/z axis. Is there a way to modify it to work on the x/y axis? Sorry my maths/trig is not quite up to the task.

Yeah doesn’t apply to IcosahedronGeometry at all

not sure wdym, but displacing any geometry with hard edges will not work.

From @Defmech

Hi, This appears to work on the x/z axis. Is there a way to modify it to work on the x/y axis? Sorry my maths/trig is not quite up to the task.

Yes, change the “swizzles” (aka .x / .y) in the displace function. You can try that here Water Glass - Shaderfrog 2.0 Hybrid Graph Demo

Double click on the vertex node:

Then change .x and .y to your preferred axes and recompile:

@Zechariah_Kapustin said:

Yeah doesn’t apply to IcosahedronGeometry at all

By default IcosahedronGeometry only has one face per side, so the vertex noise doesn’t do much for the object. If you increase the “detail” parameter it works for the geometry because there are more faces. You can test this out on the above link:

1 Like

This effect is awesome, and it looks even better applied to a transparent Three.MeshPhysicalMaterial. That’s the above link: Water Glass - Shaderfrog 2.0 Hybrid Graph Demo

It was also pretty straightforward to port this effect to Babylon :open_mouth: Glass Water - Shaderfrog 2.0 Hybrid Graph Demo

And PlayCanvas :open_mouth: Liquid Glass - Shaderfrog 2.0 Hybrid Graph Demo

It makes vertex displacement shaders pop so much more.

1 Like