Color mapping intrusion depth onto the mesh

Hello I am new to ThreeJS and I’m really eager to develop my skills in ThreeJS.
Currently, I’m trying to implement a way to color map the intrusion depth of a mesh_A(cone) into a reference mesh_B(sphere).

My objective in the below picture is to add one more layer of color gradient representing the depth the cone intruded into the sphere onto the cone surface. S.t. user can intuitively tell how much the cone intruded into the sphere.

Reading through documentation and googling some information, I think I would need to use orthographic camera. Somehow get the distance map (get distanceToSphere - distanceToCone > 0) and then render a heatmap onto the desired regions on the cone.

Would there be an example closely related to this please? I looked for one, but I couldn’t find very useful one. (Probably because I don’t have an insight yet…)

If not, could anyone suggest how I can go about followings?

  • Get the distance map
  • Apply distance map to appropriate regions of the cone faces

Thanks for the interesting question. It was really fun to try and answer it. And as a result, I made the sexiest pen in my life (so far).

The general answer is the use CSG, so you split the cone into two parts, one inside the sphere, and another outside it. The inside part is colored with a gradient texture.

However, if you do not look for a general solution, you can always figure out something else. Here is my try. You can rotate the scene for a better view of the penetration.

https://codepen.io/boytchev/full/KKxKRjv

image

4 Likes

Thank you so much for investing your time into the question. And I’m glad that you had fun while making this example. I really appreciate your help :pray:
The result is far more beautiful than I expected haha.

I have few questions to ask. It would be great if I can get your advice:

  1. If I use CSG to get the intersection and then apply the texture, wouldn’t performance drop significantly? Because I feel like getting intersection would be expensive.

  2. If there are multiple intersections because the “penetrating object” is weird looking, would it be hard to get the penetration texture offset?

  3. Do you think using DepthTexture and shader is a viable option?

  • Here is how I was attempting to implement (Struggling due to lack of knowledge & exp on ThreeJS lol)
    - Locate orthographic camera on the top (+y)
    - Take depth map of the lower hemisphere
    - Take depth map of the cone
    - subtracted_depthmap = sphere_depthmap(2d array) - cone_depthmap(2d_array)
    - filter by subtracted value > 0 (Because we want to know only the penetrating parts)
    - normalize [0~1]
    - Use fragment shader to color [0~1] mapped to RGB gradient

The idea that I implemented is for a cone penetrating a sphere towards its center. If you want to use other objects, e.g. a spoon penetrating a block of tofu, then the idea will not work. It is important for you to fix the initial conditions and then to look for the simplest solution for these conditions. Looking for the most general solution may be overkill.

  1. CSG is slower, but you can use other shapes, not just cone and sphere. There are CSG implementations that are rather fast nowadays. For some of them you can find info in this forum (e.g. three-bvh-csg)

  2. Yes, it will be harder, but you can also modify the UV coordinates of vertices, so that it looks good. The alternative is to move the coloring in a custom shader.

  3. Conceptually, your idea is OK. This is how some algorithms for shadows work. However, it will take a lot of debugging until you get it working well.

In any case, good luck with your project. I’ll be waiting for some promising results.

1 Like

That’s a nice solve @PavelBoytchev! As you’ve suggested it may be limited if switching to more complex geometries but it’s a really straight forward solution nonetheless!

@jhlee I found this example a while ago that uses a shader to calculate the difference in vertex positions of objects (if I remember correctly), updating the color (inside / outside) accordingly, it could be worth looking at this and tailoring the shader to return a gradient based on the min / max amount of difference in intersection

2 Likes

just to try to get my own head around how this could be done using a shader i decided to give it a go here…

bare in mind I pretty much know nothing about the syntax and usage of the glsl language :sweat_smile: :slight_smile: but somehow managed to modify the example shared above… the calculations are probably completely off and would likely take a lot more modification to get a working example that can be used with a variation of objects at any point in space…

EDIT: switching from a BoxGeometry to a ConeGeometry the behaviour is the same…
EDIT 2: I’ve updated the pen to include 3 geometry types which all seem to work as expected, SphereGeometry, BoxGeometry and ConeGeometry

2 Likes

What about different geometries of the penetrated object?

1 Like

good question, i was considering how that would be done, the only real way i can get my head around is with the CSG method you suggested before but instead of splitting the geometries, feeding the calculated points of intersection to the shader somehow… what are your thoughts?

this example could also extend a MeshStandardMaterial with onBeforeCompile so that standard shader chunks could be included as well as texture maps potentially but that’s way over my head for the time being

Do CSG libraries return data for intersection curves?

Another algorithm might be based on marching cubes or any other algorithm that calculates signed distances to a surface.

yeah afaik three-mesh-bvh can make these calculations, check this example for edge detection between intersected objecs, as well as this one that creates an “end cap” for the plane of intersection on a 2 million polygon model, rapidly!

ray marching may also be an option but i’d imagine that a bvh tree would yeild quicker results

2 Likes

Wow super cool!
Until now I was doing smth like below

But using three-mesh-bvh seems very promising…! I gotta look into that!

1 Like

I am shooting for the previous approach.
Currently, I got the depth map / texture of the penetrating cone as shown below. However, I am struggling to figure out how I can map this texture onto the cone WITHOUT affecting the non-penetrating regions.


^ Depth map of cone(left), face(right), cone penetrating out(below).

Below is the code to create the cone.


const customUniforms = {
  cameraNear: { value: coneDepthCamera.near },
  cameraFar: { value: coneDepthCamera.far },
  targetDepth: { value: null },
  opponentDepth: { value: null },
};
const coneGeometry = new THREE.ConeGeometry(1, 2, 32);
const coneMaterial = new THREE.MeshStandardMaterial({
  map: loader.load("https://threejs.org/examples/textures/uv_grid_opengl.jpg"),
  onBeforeCompile: (shader) => {
    // shader.uniforms.targetDepth = customUniforms.targetDepth;
    // shader.uniforms.opponentDepth = customUniforms.opponentDepth;
    shader.fragmentShader = shader.fragmentShader.replace(
      "#include <clipping_planes_pars_fragment>",
      `
        #include <clipping_planes_pars_fragment>
        uniform sampler2D targetDepth;
        uniform sampler2D opponentDepth;
        uniform float cameraNear;
        uniform float cameraFar;

        float readDepth(sampler2D depthSampler, vec2 coord) {
          float fragCoordZ = texture2D(depthSampler, coord).x;
          float viewZ = perspectiveDepthToViewZ(fragCoordZ, cameraNear, cameraFar);
          return viewZToOrthographicDepth(viewZ, cameraNear, cameraFar);
        }
      `
    );
    shader.fragmentShader = `
      #define ss(a, b, c) smoothstep(a, b, c)
      ${shader.fragmentShader}
    `.replace(
      `#include <dithering_fragment>`,
      `#include <dithering_fragment>
        float tDepth = 1.0 - (readDepth(targetDepth, vUv));
        float oppDepth = 1.0 - (readDepth(opponentDepth, vUv));
        float diffDepth = clamp(tDepth - oppDepth, 0.0, 1.0); // Penetrating depth
        vec3 diffDepthTexture = vec3(diffDepth);  // Depth texture to use

        vec3 mask = ceil(diffDepthTexture); // When there is depth info make it 1

        gl_FragColor.rgb = mix(gl_FragColor.rgb, diffDepthTexture, mask);
        gl_FragColor.a = 1.0;
      `
    );
  },
});
const coneMesh = new THREE.Mesh(coneGeometry, coneMaterial);
coneMesh.rotateX(Math.PI / 2);
coneMesh.position.set(-0.15, 2.5, -3); // S.t. it penetrates out from face forehead
coneMesh.position.z = -3;
coneMesh.layers.set(2);

scene.add(coneMesh);

Here, what I am trying to do is to use the depth info onto the faces of the cone that are penetrating outside, and keep the non-penetrating region with the original texture(uvmap texture). I am clearly doing something wrong in fragment shader. I spent quite a bit of time but not sure how to implement what I want …

Also, the targetDepth is the depth of the cone seen by the perspective camera (visualized with helper). opponentDepth is the depth of the face seen by another perspective camera (same config as the camera used to take cone depth)

And below is my render loop:

const tick = () => {
  const elapsedTime = clock.getElapsedTime();

  // Update controls
  controls.update();

  // Render to the faceTarget to use it as a texture
  renderer.setRenderTarget(faceTarget);
  renderer.render(scene, faceDepthCamera);

  // Pass tDepth to fragment shader
  facePlaneMaterial.uniforms.tDepth.value = faceTarget.depthTexture;

  // Render the cone to coneTarget
  renderer.setRenderTarget(coneTarget);
  renderer.render(scene, coneDepthCamera);
  conePlaneMaterial.uniforms.tDepth.value = coneTarget.depthTexture;

  // Pass the target and opponent depth to diff shader
  diffPlaneMaterial.uniforms.targetDepth.value = coneTarget.depthTexture;
  diffPlaneMaterial.uniforms.opponentDepth.value = faceTarget.depthTexture;

  customUniforms.targetDepth.value = coneTarget.depthTexture;
  customUniforms.opponentDepth.value = faceTarget.depthTexture;

  // Regular Render
  renderer.setRenderTarget(null);
  renderer.render(scene, camera);

  // Call tick again on the next frame
  window.requestAnimationFrame(tick);
};

tick();

To all shader experts, it would be great if I could get some aid here… :pray:

Also,
If I uncomment above two lines, which I am supposed to uncomment them, I cannot get the depth map of the cone.

Which kind of makes sense because customUniforms.targetDepth is set to null in the beginning.
However, since I set them with depthTextures in the render loop, I think I should be able to see the cone depth.

customUniforms.targetDepth.value = coneTarget.depthTexture;
customUniforms.opponentDepth.value = faceTarget.depthTexture;

How may I resolve this issue?