Mapping-fidelity in quadrilateral/triangle rendering

[vielzutun.ch]

Hi all,

I’ve investigated a behaviour which seems peculiar to me, and might benefit from an improvement. A Jsfiddle is included in this topic. The Jsfiddle goes pretty much along the lines of this Demo (geometry / teapot), except for the fact, that I’m using a LatheGeometry instead of the teapot, and a very simple 4-vertex path. You can also modify the # of lathe segments via a UI element. That said:

I’m starting with a decagon (latheSegments = 10) and am mapping the well known “uv_grid_opengl.jpg” onto it. Since this texture has 10 by 10 subdivisions, we arrive at ten nice and straight lines from each of the outer vertivces of the decagon towards the center vertex, when looking at this geometry right from the top, like so:

So far this is all nice and dandy.
But when you change the # of latheSegments to five (5), it looks like this, which is not so dandy anymore:

The vertical white texture lines at the u = 0.0, 0.2, 0.4, 0.6, 0.8 and 1.0 positions are still being rendered nicely as straight lines from the pentagon outer vertices towards the center vertex, but the lines from the 0.1, 0.3, 0.5, 0.7 and 0.9 “u”-positions are being rendered as zig-zag lines.

The zig-zag comes for a reason: each quadrilateral of a lathing band comprises of two triangles, which are being rendered independently of each other. In fact, they don’t even seem to know anything about each other. Which I believe is at the core of the problem. The vertex computation in the LatheGeometry seems impeccable, as does the (u,v) computation and assignment.

This is what appears to be happening (cyan lines added by myself for clarification):

What I would prefer to see is, that any vertical line in the texture is rendered as a straight line from the circumference (of the i.e. pentagon) towards the center vertex.

Can that be done without too much of a performance penalty?

Needless to say: the current behaviour gives really weird results with reflective materials and low path vertex and/or low lathe segment counts.

Btw: knowing what to look out for, this behaviour is also present in the teapot / geometry demo, when you select the smallest number of tessellation subdivisions (i.e: 2). See for instance the outer band of the lid.

[marquizzo]

That is a really interesting bug. It looks like the UV interpolation tries to follow parallel to the two side edges and thus ruining the diagonal.

I remember seeing something similar when dealing with reflections with a Phong material because the normals are calculated in the vertex shader, which leads to poor interpolation. Sadly, the solution to that was to add more subdivisions to make it a bit less noticeable.

This might be a WebGL quirk, and not a unique Three.js bug because converting a simple square into a trapezoid brings up the same behavior. Each triangle gets rendered independently, so it doesn’t know that it needs to compensate for the interpolation of the other. In other words, the top triangle doesn’t know that the UV at [0, 0] is stretched all the way to the left, so it doesn’t know it needs to start “leaning” that way.

You can see a live demo here:

https://jsfiddle.net/marquizzo/c1pvwdbg/

[vielzutun.ch]

As I was waiting for answers/opinions to pour or trickle in ( thanks, @marquizzo ), I researched a little more on this issue. It turns out, I’m not the first one to notice this behaviour. And it’s not confined to three.js either.

Wikipedia calls this

Texture mapping | Affine texture mapping

and it’s being attributed as “the fastest form of texture mapping”. So we’re not talking about a genuine “bug”, but rather about a “known limitation”. My wish for a higher fidelity mapping isn’t original either: Wikipedia goes on to elaborate about

Texture mapping | Perspective correctness

The most promising parts of the latter section seem to be, that:

• doing this in software is about 16 times as expensive as affine texture mapping
• 3D graphics hardware typically supports perspective correct texturing.

Note also in the section on Hardware implementations:

Texture mapping - Wikipedia
Modern graphics processing units (GPUs) provide specialised fixed function units called texture samplers , or texture mapping units , to perform texture mapping, usually with trilinear filtering or better multi-tap anisotropic filtering and hardware for decoding specific formats such as DXTn. As of 2016, texture mapping hardware is ubiquitous as most SOCs contain a suitable GPU.

So this does actually sound quite promising to me.

The answer to my original question: “can it be done without too much performance penalty?” seems to be a clear Yes, it can!

The more specific question now is:

Can it be unleashed for use in three.js via firmware, driver, configuration switch or else?

[marquizzo]

I believe you’re confusing the distortion we’re seeing with perspective-corrected texture interpolation. Perspective-correction is what the Wiki article is talking about, and it’s something that already happens automatically in WebGL:

“As of 2016, texture mapping hardware is ubiquitous as most SOCs contain a suitable GPU.”

I’ve edited my working example to demonstrate the difference.

• The top plane tilts backwards to create perspective, and you can see the GPU automatically adjusts for this when interpolating the texture.
• The bottom plane doesn’t tilt, but instead stretches its base to create a trapezoid. Here we can see the issue where the UV stretching creates the distortion along the diagonal.

[vielzutun.ch]

Yes, for the time being, I’ve come to the same conclusion.

Even though “Perspective Correct Texturing” for rectangles involves a perspective division (only). And I can’t really believe, that all a dedicated hardware texturing unit of a current GPU does is substitute one line of code in silicon.

This is a very enlightening introduction into WebGL and Perspective Correct (and incorrect!)Texturing, including Jsfiddle-like live and running demos (at about 2/3s down the page):

WebGL 3D Perspective Correct Texture Mapping

I’ve yet to find a good introduction into how hardware texture units work and how they can be addressed.

[prisoner849]

If you want to have more or less correct texturing for 5-segment LatheGeoemtry, then double the segments (10) and interpolate every 2nd set of points in between of 1st and 3rd, over 2 sets: Edit fiddle - JSFiddle - Code Playground

function beautifyLathe(g){
	
  let pos = g.attributes.position;
  
  let pCount = g.parameters.points.length;
  let segs = g.parameters.segments;
  
  let pStart = new THREE.Vector3();
  let pEnd = new THREE.Vector3();
  let pMid = new THREE.Vector3();
  
  for(let i = 0; i < segs; i += 2){
  	for(let p = 0; p < pCount; p++){
  	  pStart.fromBufferAttribute(pos, pCount * i + p);
      pEnd.fromBufferAttribute(pos, pCount * (i + 2) + p);
      pMid.addVectors(pStart, pEnd).multiplyScalar(0.5);
      pos.setXYZ(pCount * (i + 1) + p, pMid.x, pMid.y, pMid.z);
    }
  }
  
  g.computeVertexNormals();
  
}

[vielzutun.ch]

That is a neat idea for a very specific use case - thank you.

To be honest, I was fitting the # of lathe segments in my example to the # of vertical lines of that specific sample texture, to show the effect in maximum clarity. Actually, the kink in mapping a straight vertical line to a quadrilateral will be seen for any number of lathe segments which is different from an exact integer multiple of the number of visible texture map subdivisions, i.e. vertical white lines.

Ultimately I’m aiming at getting more realistic reflective mapping of natural textures, which typically lack any regular vertical traits. So the nature and cause of the problem would have been much more difficult to illustrate.

[marquizzo]

I found a solution! I asked at GameDev.StackExchange, which led me to a really good answer.

You know how you can fix perspective texturing by adding a 4th component w to the xyz position calculation? Turns out you can do the same to your uvs and turn them into uvws, where the w component contains the scaling factor of how stretched the texture is. It would require custom GLSL shader code injection to be used with Three.js materials, but turns out it is possible:

How can I fix zig-zagging UV mapping artifacts on a generated mesh that tapers?

[vielzutun.ch]

@marquizzo

I found a solution!

That sounds very promising indeed!

Thanks for the follow-up. I’ll take a deeper look into this tomorrow …

[vielzutun.ch]

OK, so here’s my findings:

The GOOD:
The proposed solution works for trapezoidal quadrilaterals, in that it eliminates the “kink” along any of the diagonals.

The BAD:
The appearance of the trapezoid resembles perspective correct texturing, in that the rows of the regular checkerboard become narrower, going from the bottom edge towards the top. Although all vertices are in the xy-plane having z= 0.

The UGLY:
This solution does not work for arbitrary quadrilaterals, without any parallel edges. I may have lacked the patience to experiment with different scaling factors which might have to be applied along two axes, as opposed to one “scale” factor to be applied to the vertices of one of the parallel edges of a trapezoid.

A complete Jsfiddle is provided, which I forked from a working example of a spinning, perspective correctly textured cube on

webglfundamentals.org/webgl/lessons/webgl-3d-perspective-correct-texturemapping dot html

, which I stripped down to one still quadrilateral. After which I applied the proposed changes regarding three-component uvws.

https://jsfiddle.net/Chrisssie/9e1s5dwp/

[marquizzo]

@vielzutun.ch

The appearance of the trapezoid resembles perspective correct texturing

But that’s exactly what we were trying to achieve! In the context of the rest of the LatheGeometry it’ll make sense to our eyes. It’ll end up looking exactly like @prisoner849’s image:

[vielzutun.ch]

@marquizzo

But that’s exactly what we were trying to achieve!

Not quite, I’m afraid. In the context of my previous Jsfiddle ( all four vertices having their z-component = 0.0), I’d expect equal spacing in y-direction, while proportional (to the taper) spacing in x-direction. Without any kinks in any of the diagonals, of course.

Any perspective foreshortening would have to come on top of that, if applicable due to perspective, but not right from the start when viewing straight-on.

[prisoner849]

Maybe bilinear interpolation will help, I’m not sure though.
I tried it here: Trilinear interpolation of vertices, see the codepen in the PS. But it was made in vertex shader with a segmented plane.

[vielzutun.ch]

Well, a good night’s of sleep does make a difference

Linear spacing along v, while proportional spacing along u.

The decisive change:

In the fragment shader, apply the division by scale only to the u-component, while leaving the v-component unchanged:

void main() {
   gl_FragColor = texture2D(u_texture, vec2( v_texcoord.x / v_texcoord.z, v_texcoord.y ) );
}

Plus, when setting up the array with uvws, don’t scale the vs in the first place.

I’ve updated the Jsfiddle:

https://jsfiddle.net/Chrisssie/9e1s5dwp/

P.S.: now that I know how to separate the effects of proportional scale of each axis from that of the other axis, I will experiment with bilinear scaling for arbitrarily shaped quadrilaterals. But don’t hold your breath …

[vielzutun.ch]

Here’s the result from my

@vielzutun.ch

experiment with bilinear scaling for arbitrarily shaped quadrilaterals =>

It’s not really an “irregular” quadrilateral yet, but has no parallel edges nevertheless. So I’m confident it could be done for a truly irregular quadrilateral as well. A little bit of context:

I started out with a unit square, centred about the coordinate origin, then moved vertex #4 out by +0.5 units in each direction:

                      Y
                      |
              [2]-----|-----[4]
               |      |      |
               |      |      |
               |       -----------> X
               |     /       |
               |    /        |
              [1]--/--------[3]
                  /
                 Z
 
 */
function setGeometry(gl) {
  var positions = new Float32Array(
    [
    // front face, lower-left triangle
    -0.5, -0.5,  0.0,  // #1
     0.5, -0.5,  0.0,  // #3
    -0.5,  0.5,  0.0,  // #2
    
     // front face, upper right triangle
    -0.5,  0.5,  0.0,  // #2
     0.5, -0.5,  0.0,  // #3
     1.0,  1.0,  0.0,  // #4 make the the square irregular (no parallel edges), by moving this vertex out
    ]);
  gl.bufferData(gl.ARRAY_BUFFER, positions, gl.STATIC_DRAW);
}

Moving vertex #4 out yields two vanishing points at a distance of 3.0 units from the lower-left vertex of the quadrilateral:

The maximum inclination (red) of the left fan-lines is 1/3, while the minimum inclination (blue) of the bottom fan-lines is 3/1, with “inclination” being the value “m” in the straight line equation:

y = m * x + b

I was unable to realise this with the 3-component uvw-approach and the “scale” - “scale back” trick as before. So I reverted this back to the familiar 2-component uv-approach. The whole magic is in the fragment shader, which now looks like this:

void main() {
   gl_FragColor = texture2D( u_texture, vec2( 
   	  3.0 * v_texcoord.x / (v_texcoord.y + 3.0),
      3.0 * v_texcoord.y / (v_texcoord.x + 3.0) ) );
}

A new Jsfiddle is provided:

https://jsfiddle.net/Chrisssie/vnc2pg6w/

Looking back, this was an interesting, yet imo futile exercise as far as advancing the fidelity of texturing in Three.js goes. Because there are no procedurally generated geometries in Three.js which result in irregular quadrilaterals that I’m aware of.

Thinking of it, the rectangle/trapezoid case has the most potential in that regard. Because

• Cone
• Cylinder
• Extrude
• Lathe
• Plane
• Ring
• Sphere
• Torus
• TorusKnot
• Tube

all produce quadrilaterals, which are either rectangular or trapezoidal, safe for the bands bordering poles.

I think I’m going to try my hand at the Lathe geometry and see how far I’ll get …

[vielzutun.ch]

@vielzutun.ch

I’m going to try my hand at the Lathe geometry and see how far I’ll get …

I have a working copy on my local computer, a Proof of Concept at this stage and for a LatheGeometry only.
I’d like to publish how it’s done, in a Jsfiddle. Unfortunately I’m running into unexpected Problems and am asking for help on this.

So, currently I have a teaser video only in place of a genuine proof:

[vielzutun.ch]

Thanks to @repalash , here we go:

See the effect of non-affine, perspective correct texturing (no more zig-zag lines in the diagonals of quadrilaterals) integrated into Three.js: https://jsfiddle.net/Chrisssie/pto8v03z/

Disclaimer: This is currently a Proof of Concept only, and has been implemented for a LatheGeometry ONLY. Expect side effects for different geometries and/or non-MeshPhongMaterials.

I changed about six LOC in the shaders, and around the same number in the LatheGeometry function.

All changed lines of code have been marked with a ‘////’ signature at their ends.
View the changes in a hacked copy of three.module.js which is imported in the Jsfiddle.

Compare to the current behaviour(same link as initial post): https://jsfiddle.net/Chrisssie/zgkx7bso/

[marquizzo]

Are you sure you fixed the CORS issue? I’m getting this error:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://vielzutun.ch/wordpress/public/three.module.js. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).

[vielzutun.ch]

@marquizzo

Are you sure you fixed the CORS issue? I’m getting this error:

Thanks for your feedback - I’ll look into this.

[vielzutun.ch]

@marquizzo

Are you sure you fixed the CORS issue?

@marquizzo confirmed that the Jsfiddle now works

Play with the “latheSegments” slider and prepare to be stunned!

https://jsfiddle.net/Chrisssie/pto8v03z/