Adding ID Color maps to MeshStandardMaterial in three.js

Questions
,
1

I would like to add color ID maps to three.js, similar to the technique available in Substance Painter:

The result must also be composited with the MeshStandardMaterial, such that the color map is added, then multiple layers of id maps are applied on top of that. (then the subsequent PBR elements added as standard).

I envisage that up to 4 id maps could be contained (per channel) in a single jpg image. Each channel being the alpha value of the applied color where 0x00 is invisible and 0xFF is fully visible. A color parameter would then be passed (per channel) and applied to the fragment shader, altering the color at runtime.

Here is some pseudo code for the API that might make sense:

// load an id map with 4 separate channels
IdMapTexture texture = (IdMapTexture)mTextureLoader.load(path, callback);

MeshStandardIdMapMaterial material = new MeshStandardIdMapMaterial();
material.idMaps[0] = texture;

// apply colors to the channel via the fragment shader
material.idMaps[0].color0 = new Color(0x336699);
material.idMaps[0].color1 = new Color(0xFF0099);
material.idMaps[0].color2 = new Color(0x640971);
material.idMaps[0].color3 = new Color(0x216647);

// apply the material to the mesh
mesh.material = material;

My question boils down to 3 main topics (Highest == best).

  1. Is there anything like this already available?
  2. Is there any compositing technique that could make building this more structured/simplistic?
  3. Is there a standard way/tools for extending MeshStandardMaterial for three.js?

Regards

2

This is just using colors in a texture to assign material ids, right?

This is definitely useful in a modelling / material design program (such as Painter), but I would expect that you’d bake the ids into the model to assign materials to each polygon before importing it into three.js, rather than using an additional texture slot.

3

The point is to add layers and change values at runtime without creating duplicate geometry for a clothing design tool where the layers are designed by artists. I waited hours for discourse to send me a welcome email and then got insta-banned when I made my post so I have actually already got pretty far with this by doing the following:

I added glsl vertex and fragment shaders into the html document along with the glsl mix() function to add multiple transparent pngs. The pngs were loaded with the three.js texture loader system and passed to the fragment shader.

In the fragment shader I subsequently set the color of the fragments directly using a vec3. It is important to note that I needed to set the material transparent property to true and the blend function accordingly.

I intend to compress data into channels tomorrow and then add the lights and PBR equations again, but I think I got it already. Cheers

4

gpaluk, I think you are going to want to use a 2d canvas for a lot of this. I’m just about out of gas tonight (brutal day), and I’m not sure exactly what all you are trying to accomplish, but I have a feeling 2d canvas is where you should be looking. I also use Painter and pbr materials extensively, and I am able to do everything I need with Standard Mat as is, in conjunction with dynamic 2d canvases and lots of js… I’d be very interested if you can cook up some stuff that would enhance pbr in three, I just can’t imagine it at the moment. :wink:

5

Yes, that’s correct, I will use 2D canvas for text and loading decal images, but some are also produced by our artists. In fact I already made an example of canvas2d text :slight_smile:

6

You really need to look deeper into what you can do with dynamic 2d canvases with pbr mats in three. I have to shut down and chill for the remainder of this oh-so-long-and-crazy-day. If you think it is limited to text and decals, you are woefully ill informed. :stuck_out_tongue:

7

Not ill informed, just know my requirements. :slight_smile: Feel free to elucidate though.

8

@gpaluk I managed to solve this using a CanvasTexture. By compositing the materials with the Color ID map in a custom shader and drawing to an empty canvas. Then using that canvas as a texture in THREEJS.

its been a long time since this thread was first started. But if you get this message. I’ll be happy to share my implementation with you.

9

The basic fragment shader required is as shown below:

this is native opengl with glsl version 330 but not difficult to rework to webgl and glsl available on webgl

#version 330;

in mediump vec2 vert_uv;

uniform sampler2D idmap_texture;

uniform sampler2D input_textures[3];

out mediump vec4 FragColor;

mediump vec4 texid_texture(sampler2D id, mediump vec2 uv, sampler2D r, sampler2D g, sampler2D b) {
  mediump vec4 col_id = texture(id, uv);
  mediump vec4 r_sample = texture(r, uv);
  mediump vec4 g_sample = texture(g, uv);
  mediump vec4 b_sample = texture(b, uv);

  mediump vec3 r_contrib = mix(col_id.rrr, r_sample.rgb, col_id.r);
  mediump vec3 g_contrib = mix(col_id.ggg, g_sample.rgb, col_id.g);
  mediump vec3 b_contrib = mix(col_id.bbb, b_sample.rgb, col_id.b);

  return vec4(r_contrib + g_contrib + b_contrib, 1.0);
}

void main() {

  mediump vec4 id_map = texture(idmap_texture, vert_uv);
  mediump vec4 output = texid_texture(idmap_texture, vert_uv, input_textures[0], input_textures[1], input_textures[2]);

  FragColor = output;
}