Fragment Shader Performance Question

[titansoftime]

So I Implemented a texture splatting technique which I found in this very forum.

How to create a multiple textured terrain

Thanks for sharing your code! Hint: When you post code in this forum, please use backticks in order to open and close a code section. This will improve formatting and readability. I’ve edited your post so you can see how it’s done.

Basically I am replacing the default map_fragment shader code with the following:

'vec4 tbBlend=texture2D( texBlendMap, vUv );',

'float tbBaseWeight=1.0 - max(tbBlend.r, max(tbBlend.g, tbBlend.b));',

'vec4 base =  tbBaseWeight * texture2D( texBaseColor, vUv * repeatBase );',
'vec4 color1 = tbBlend.r * texture2D( texTile1Color, vUv * repeatTile1 );',
'vec4 color2 = tbBlend.g * texture2D( texTile2Color, vUv * repeatTile2 );',
'vec4 color3 = tbBlend.b * texture2D( texTile3Color, vUv * repeatTile3 );',
'vec4 newColor = (vec4(0.0, 0.0, 0.0, 1.0) + base + color1 + color2 + color3) / (tbBaseWeight+tbBlend.r+tbBlend.g+tbBlend.b);',

'newColor.a = tbBlend.a;',

'diffuseColor = mapTexelToLinear( newColor );',

This produces exactly what I want, however on mobile the performance is pretty bad. What would otherwise be a 60fps scene will drop to 20fps and my phone gets HOT.

I was a bit surprised to see such a strong impact, on the other hand, shader programming is not something I have a ton of experience with.

Is there something wrong with this shader code? Is there a way to optimize it? I really like the result as I can simply paint trails and such, but ouch, the performance hit is just too strong.

Any help would be greatly appreciated.

[Mugen87]

@titansoftime

Is there a way to optimize it?

How big is the resolution of your used textures?

[titansoftime]

On mobile all textures default to 512x512.

The blendmap was originally 2048x2048, so I figured that may be the issue. Changed that to 512x512 as well, no difference =[

[Mugen87]

Yeah, texture resolution can be an issue on mobile GPUs due to the limited main memory bandwidth. Do you still see no performance improvement if you resize the textures even more? E.g. to 64x64? I know it’s not a usable size it’s just for testing^^.

BTW: What mobile device are you using?

[titansoftime]

Yea, reducing to 64x64 for the actual textures made a difference. Lowering anisotropic filtering seems to help a bit too. Not quite the 55-60fps that I get when using my prior method (painting materials using a “materialmap” (128x128 png and use red value for texture id) and assigning textures based on materialindex (for this I use regular Geometry, set the face materialIndex and then later convert to BufferGeometry)), but much better (45-52 fps compared to 27-32 fps).

My old method can use 1024x1024 textures though, no problem.

I am testing on a Samsung Galaxy s7 edge in Chrome.

[titansoftime]

I built a reproduction of this issue.

I initially tried it through js fiddle, but the performance seemed fine. I thought maybe the smaller viewport may be why so I checked the full screen result and it was also ok. However on the fullscreen link, I noticed that everything was very blurred and somewhat pixelated (on mobile at least), similar to how it looks when you set your viewport setting wrong.

So then, I simply copy pasted the jsfiddle code into an html file and the issue presented itself.

I’ve noticed that the performance gets worse when looking more towards the direction of the nearby “water”. Weird.

Also as it always seems, the first 30 seconds to minute after load seem to run ok, then perf tanks. This is something I always have noticed though. No matter what, first minute or so after load runs fantastic. Probably some mobile browser throttle mechanism.

Anyway, this is what I’ve made:

jsfiddle (performance seems ok): https://jsfiddle.net/titansoftime/kewgm30c/

fullscreen jsfiddle (performance seems ok): https://jsfiddle.net/titansoftime/kewgm30c/embedded/result

html (issue is noticable): https://www.titansoftime.com/perftest.html

Increasing detail to 1024 and anisotropy to 4 makes it quite noticeable (at least on my mobile device). Actually it seems that anisotropy has the largest impact.

Reducing splat texture resolution, disabling the splat textures and even setting the character mesh skinning to false (oddly enough) all seem to bring the example to a steady 60 fps at all angles.

Wish I could explain this =/ I really want to be able to have my terrain “paintable” =[

[Mugen87]

@titansoftime

Increasing detail to 1024 and anisotropy to 4 makes it quite noticeable (at least on my mobile device).

On my Pixel 1 (October 2016) and latest Chrome (73.0.3683.90) and Android 9, https://www.titansoftime.com/perftest.html runs at 60 FPS. The Samsung Galaxy S7 Edge is a bit older but has comparable hardware (Qualcomm Snapdragon 820 vs. Snapdragon 821 from the Pixel, same RAM). I’m surprised about that performance differences.

Can you make a test with a different mobile device?

[Usnul]

Tried on desktop (GTX 1080), the performance is really nice:

(https://www.titansoftime.com/perftest.html)

[titansoftime]

Ha yea it runs great on my gtx 1060 as well =]

I’ve actually rewrote the shader code (in app) to reduce the amount of textures pushed and to only utilize textures that need to be used. Helped a good bit.

One of the mode obvious optimizations I added was to just use the one terrain plain instead of overlapping the two. I only did this originally to workaround abrupt texture “seams” where geometry faces would use different material indexes. With this splatting/painting method, this is no longer necessary.

Performance is at least acceptable (35-50fps) on mobile now in app (which is good because I’ve got reflective water, instanced grass, and like 50 skinned meshes per scene, not to mention a whole lot of other things going on (web socket updates etc.).

At this point I really only have one question: Why in the world does this splat material require a highp precision? Mediump of course works fine on the prior method of using terrain faces with different material indexes and a standard array of materials. All this is doing is fetching a couple additional textures and doing some additional vec4 math right?

[Usnul]

I don’t know about highp really, I use medium in my implementation without any issues.

Out of curiosity, what’s the typical resolution on your terrain?
both for height map and for the polycount

For my game it’s currently it’s 512x512 heightmap and mesh resolution of 675x675 quads or 911,250 triangles (it’s broken up into sections).

[titansoftime]

Keep in mind, most (if not all) desktops run highp no matter what, even if you set mediump. On mobile, it will use whatever you specify (if it’s able).

Still confused as to why this shader code breaks on mediump…

tileblend_fragment: [

	'vec4 tbBlend;',
	
	'float tbBaseWeight;',

	'if ( tileCount > 1 ){',

		'tbBlend = texture2D( texBlendMap, vUv );',

		'tbBaseWeight = 1.0 - max(tbBlend.r, max(tbBlend.g, tbBlend.b));',											
		
		'vec4 texelColor = vec4(0.0, 0.0, 0.0, 1.0);',
		
		'texelColor = mix(texelColor,texture2D( map, vUv * repeatBase ), tbBaseWeight);',
		
		'texelColor = mix(texelColor,texture2D( texTile1Color, vUv * repeatTile1 ), tbBlend.r);',						

		'if( tileCount > 2 ) {',
		
			'texelColor = mix(texelColor,texture2D( texTile2Color, vUv * repeatTile2 ), tbBlend.g);',

		'}',							

		'if( tileCount > 3 ) {',
		
			'texelColor = mix(texelColor,texture2D( texTile3Color, vUv * repeatTile3 ), tbBlend.b);',

		'}',											

		'texelColor.a = tbBlend.a;',

		'diffuseColor = mapTexelToLinear( texelColor );',
		//'diffuseColor = texelColor;',

	'} else {',

		'vec4 texelColor = texture2D( map, vUv );',

		'texelColor = mapTexelToLinear( texelColor );',

		'diffuseColor *= texelColor;',

	'}',

].join('\n'),

highp:

mediump:

I tried googling an answer but honestly don’t even know enough in order to formulate an intelligible question.

Regarding the performance variance… I’ve investigated for weeks now. It seems to be completely unpredictable. Code changes just bring up red herrings. Chrome mobile browser itself has rapid swings in performance whereas android webview (using chrome as renderer) is totally fine. I’m throwing up my hands and am going to label this a browser issue. I can’t take it anymore, losing my mind, must move on.

To answer your questions Unsul, terrain heightmap resolution is 128x128 (image is actually 129x129 to workaround the removal of quads back in r60) for a total of 32,768 triangles (contiguous).

Terrain texure resolution defaults to 512x512 on mobile, 1024x1024 on non-mobile. Mobile can go up to 1024x1024 and desktop can go up to 2048x2048.

[marquizzo]

This is a shot in the dark, but have you tried making one big texture made up of 2x2 small textures, instead of sampling from 4 different ones? Something like this:

-------------
|  1  |  2  | 
-------------
|  3  |  4  | 
-------------

• Create a 1024x1024 texture from four 512x512 textures.
• Scale your UVs down to the [0.0, 0.5] range instead of [0.0, 1.0]
• Sample the same texture four times:

uv + vec2(0.0, 0.0);
uv + vec2(0.5, 0.0);
uv + vec2(0.0, 0.5);
uv + vec2(0.5, 0.5);

I don’t know if this will fix your problem, but worth a shot!

[titansoftime]

Now that is interesting!

I’m wondering, will this work on desktop as well for folks who choose 2048x2048 resolution? Would stitching together a 4096x4096 have any adverse effects compares to 4 separate 2048 textures? This would make parallel downloading better for initial load I think.

Either way, I totally am going to try this, if anything just for the fun of it =]

Thanks for the idea!

[Usnul]

I’m curious what the result would be too. It might be the texture switching.
@titansoftime it looks like your UVs are getting clamped (on mediump), maybe have a look at the UV calculation logic?

[titansoftime]

OK, kinda got this working in a simple form. One thing I cannot for the life of me figure out though is how to get the textures to repeat correctly. The way this is set up, I have no idea how to pull it off =[

Edit fiddle - JSFiddle - Code Playground

Is it even possible?

maybe have a look at the UV calculation logic?

I’ll take a stab at it, but have no idea why it would work on highp but not mediump. Perhaps I need to fix the resulting uv coords to a lower precision value? Only thing I can think of. Maybe some values are something like vec2(0.234, 0.8676765354767387827537…) or something /shrug

[titansoftime]

It is indeed possible (repeating only part of a texture), but of course not without it’s side effects. Now I am getting a quite noticeable black seam between “tiles”. I was able to mitigate this to a point by setting the min and mag texture filter to LinearFilter, but it’s still somewhat noticeable and also kills performance (maybe, jsfiddle is a bit unpredictable). Not to mention this filter kinda changes the appearance of the plane a bit.

It’s possible I simply was off a pixel when combining the textures in photoshop. Will have to check when I get back home.

But hey, still cool stuff!

https://jsfiddle.net/titansoftime/o674d2qh/

[Usnul]

@titansoftime

Now I am getting a quite noticeable black seam between “tiles”.

There’s actually a lot of literature on this subject. Most common way is to pad the patch, to make sure that mipmap doesn’t contain texels from other patches as allow texture filtering. Maybe it could be interesting to collaborate a bit on our terrain engines. Drop me a message if you’re interested.