InstancedMesh + implement billboard

I have deployed an object using an InstancedMesh .
But it is difficult to apply the billboard.

I have deployed an object using an InstancedMesh .
But it is difficult to apply the billboard.
When I put the billboard formula in the shader, only one object is displayed.
How can I display all objects?

code is =>

// import * as THREE from ‘https://cdn.jsdelivr.net/npm/three@0.131.3/build/three.module.js’;

import {OrbitControls} from 'https://cdn.skypack.dev/three@0.131.3/examples/jsm/controls/OrbitControls.js';

let instances = 100;

let container = document.querySelector( '[name=container]' );

let scene = new THREE.Scene();

scene.background = new THREE.Color( 0x999999 );

let camera = new THREE.PerspectiveCamera( 50, window.innerWidth / window.innerHeight, 1, 1000 );

camera.position.set(0, 10, 10);

camera.lookAt( new THREE.Vector3( 0, 1, 0 ) );

//  --- ---     billboard material

class Billboard extends THREE.MeshBasicMaterial {

    constructor( _customCreateOption ){

        let baseCreateOption = {

            onBeforeCompile: shader =>{

                shader.vertexShader = shader.vertexShader.replace(

                    `#include <project_vertex>`,

                    `

                    vec4 mvPosition = vec4( transformed, 1.0 );

                    #ifdef USE_INSTANCING

                        mvPosition = instanceMatrix * mvPosition;

                    #endif

                    mvPosition = modelViewMatrix * mvPosition;

                    // gl_Position = projectionMatrix * mvPosition;

                    gl_Position = projectionMatrix * (modelViewMatrix * vec4(0.0, 0.0, 0.0, 1.0) + vec4(position.x, position.y, 0.0, 0.0));

                    `

                    );

                console.log("shader?", shader.vertexShader);

            }

        };

        super( Object.assign({},baseCreateOption,_customCreateOption) );

    }

};

//  --- ---     geometry

class TerrainGrass extends THREE.InstancedMesh {

    constructor(){

        //  --- ---     geometry

        const geometry = new THREE.InstancedBufferGeometry();

        const vertexBuffer = new THREE.InterleavedBuffer( new Float32Array( [

            // - 1, 1, 0, 0, 0, 0, 0, 0,

            // 1, 1, 0, 0, 1, 0, 0, 0,

            // - 1, - 1, 0, 0, 0, 1, 0, 0,

            // 1, - 1, 0, 0, 1, 1, 0, 0,

            - 0.3, 0.3, 0, 0, 0, 0, 0, 0,

            0.3, 0.3, 0, 0, 1, 0, 0, 0,

            - 0.3, - 0.3, 0, 0, 0, 1, 0, 0,

            0.3, - 0.3, 0, 0, 1, 1, 0, 0,

        ] ), 8 );

        const positions = new THREE.InterleavedBufferAttribute( vertexBuffer, 3, 0 );

        geometry.setAttribute( 'position', positions );

        const uvs = new THREE.InterleavedBufferAttribute( vertexBuffer, 2, 4 );

        geometry.setAttribute( 'uv', uvs );

        const indices = new Uint16Array( [

            0, 2, 1,

            2, 3, 1,

        ] );

        geometry.setIndex( new THREE.BufferAttribute( indices, 1 ) );

        //  --- ---     material

        const material = new Billboard();

        // material.map = new THREE.TextureLoader().load( 'Mickey_Mouse.png' );

        material.map = new THREE.TextureLoader().load( 'https://i.imgur.com/b6LjMuN.png' );

        material.map.flipY = false;

        // material.blending = THREE.AdditiveBlending;

        material.alphaTest = 1;

        // per instance data

        const matrix = new THREE.Matrix4();

        const offset = new THREE.Vector3( 0, 0, 0 );

        const orientation = new THREE.Quaternion();

        const scale = new THREE.Vector3( 1, 1, 1 );

        // let mesh = new THREE.InstancedMesh( geometry, material, instances );

        // mesh.setMatrixAt( 0, matrix );

        super( geometry, material, instances );

        this.setMatrixAt( 0, matrix );

        // to deploy

        this.deploy();

    }

    deploy(){

        // per instance data

        const matrix = new THREE.Matrix4();

        const offset = new THREE.Vector3();

        const orientation = new THREE.Quaternion();

        const scale = new THREE.Vector3( 1, 1, 1 );

        for( let i=0;i<instances;i++ ){

            let x = i % 10 - 5;

            let z = Math.floor(i/10) - 5;

            offset.set( x, 0, z );

            matrix.compose( offset, orientation, scale );

            this.setMatrixAt( i, matrix );

        }

        this.instanceMatrix.needsUpdate = true;

        

    }

}

// scene.add( mesh );

let mesh = new TerrainGrass();

mesh.position.set( 0, 0.3, 0 );

scene.add( mesh );

scene.add( new THREE.GridHelper(10, 10, "white", "white") );

let renderer = new THREE.WebGLRenderer();

renderer.setPixelRatio( window.devicePixelRatio );

renderer.setSize( window.innerWidth, window.innerHeight );

container.appendChild( renderer.domElement );



// orbitcontrols

let controls = new OrbitControls( camera, renderer.domElement );

function animate() {

    requestAnimationFrame( animate );

    render();

}

function render() {

    renderer.render( scene, camera );

}

animate();

I dont know how use intancedmesh. Try InstancedBufferGeometry. And add there attribute “offset”.
https://threejs.org/examples/webgl_buffergeometry_instancing.html
And you can animate sprite texture.

The code I referenced =>
https://threejs.org/examples/#webgl_buffergeometry_instancing_interleaved

Only front is used here.
I want this side to always face the camera.
It will be used as a terrain.

1 Like

Any reference picture/video of the desired result?

The first picture is desired result.
I want each object to have a billboard applied to it.

I don’t know how to put each instance value in the billboard formula below.
gl_Position = projectionMatrix * (modelViewMatrix * vec4(0.0, 0.0, 0.0, 1.0) + vec4(position.x, position.y, 0.0, 0.0));

I was implementing a stretched billboard renderer.
It is often used in lightning and fire effects.
It’s more difficult than I expected.

about video

Need points geometry but there size limit is 512 or sprites.
I am using my InstancedBufferGeometry like sprites rotation and sorting by distance to camera because of transparent “bug”.

PointGeometry is also being tested.
PointGeometry has the disadvantage of not being able to use objects.

Terrain needs to put trees or grass, so it needs to be able to spray objects.

Did you put it as an image? Did you specify all the vertices?

It like merged sprites but modified

var geometry=new THREE.InstancedBufferGeometry();
geometry.setAttribute('position',new THREE.Float32BufferAttribute(new Float32Array([-0.5,0.5,0,-0.5,-0.5,0,0.5,0.5,0,0.5,-0.5,0,0.5,0.5,0,-0.5,-0.5,0]),3));
geometry.setAttribute('uv',new THREE.Float32BufferAttribute(new Float32Array([0,1,0,0,1,1,1,0,1,1,0,0]),2));
geometry.setAttribute('offset',new THREE.InstancedBufferAttribute(new Float32Array(),3));
geometry.setAttribute('scale',new THREE.InstancedBufferAttribute(new Float32Array(),2));
geometry.setAttribute('quaternion',new THREE.InstancedBufferAttribute(new Float32Array(),4));
geometry.setAttribute('rotation',new THREE.InstancedBufferAttribute(new Float32Array(),1));
geometry.setAttribute('color',new THREE.InstancedBufferAttribute(new Float32Array(),4));
geometry.setAttribute('blend',new THREE.InstancedBufferAttribute(new Float32Array(),1));
geometry.setAttribute('texture',new THREE.InstancedBufferAttribute(new Float32Array(),1));


mat["sprite"]=new THREE.ShaderMaterial({
uniforms:{
map:{value:[tex["cloud"],tex["sprite_yellow"],tex["smoke"],tex["fire"],tex["beam"],tex["flare_blue"],tex["glass_1"],tex["window"]]},
cameraAngle:{value:[0,0]},
time:{value:0}
},
vertexShader:vs["sprite"],
fragmentShader:fs["sprite"],
side:THREE.DoubleSide,
transparent:true,
depthWrite:false,
blending:THREE.CustomBlending,
blendEquation:THREE.AddEquation,
blendSrc:THREE.OneFactor,
blendDst:THREE.OneMinusSrcAlphaFactor
});


mesh["sprite"]=new THREE.Mesh(geometry,mat["sprite"]);
mesh["sprite"].frustumCulled=false;
mesh["sprite"].matrixAutoUpdate=false;
mesh["sprite"].updateMatrixWorld=function(){};
scene.add(mesh["sprite"]);
vs["sprite"]=`


attribute vec3 offset;
attribute vec2 scale;
attribute vec4 quaternion;
attribute float rotation;
attribute vec4 color;
attribute float blend;
attribute float texture;
uniform float time;
varying vec2 vUv;
varying vec4 vColor;
varying float vBlend;
varying float num;
uniform vec2 cameraAngle;


void main(){


float angle=time*rotation;


vec3 vRotated=vec3(position.x*scale.x*cos(angle)-position.y*scale.y*sin(angle),position.y*scale.y*cos(angle)+position.x*scale.x*sin(angle),position.z);


vUv=uv;
vColor=color;
vBlend=blend;
num=texture;


vec3 vPosition;


// ПОВОРОТ В СТОРОНУ КАМЕРЫ КАК ОБЫЧНЫЙ СПРАЙТ
if(quaternion.w==2.0){
vec3 cameraRight=vec3(viewMatrix[0].x,viewMatrix[1].x,viewMatrix[2].x);
vec3 cameraUp=vec3(viewMatrix[0].y,viewMatrix[1].y,viewMatrix[2].y);
vPosition=(cameraRight*vRotated.x)+(cameraUp*vRotated.y);
}
// ПОВОРОТ В СТОРОНУ КАМЕРЫ ПО XZ ПО-ПРОСТОМУ
else if(quaternion.w==3.0){
vPosition=vRotated;
float angleXZ=cameraAngle.y;
vPosition.x=cos(angleXZ)*vRotated.x-sin(angleXZ)*vRotated.z;
vPosition.z=cos(angleXZ)*vRotated.z+sin(angleXZ)*vRotated.x;
}
// ПОВОРОТ В СТОРОНУ КАМЕРЫ ПО XZ ПО-СЛОЖНОМУ В ЗАВИСИМОСТИ ОТ ПОЗИЦИИ
else if(quaternion.w==4.0){
vPosition=vRotated;
float x=cameraPosition.x-offset.x;
float z=cameraPosition.z-offset.z;
float angleXZ=acos(z/sqrt(x*x+z*z));
if(x>0.0){ angleXZ*=-1.0; }
vPosition.x=cos(angleXZ)*vRotated.x-sin(angleXZ)*vRotated.z;
vPosition.z=cos(angleXZ)*vRotated.z+sin(angleXZ)*vRotated.x;
}
// ПОВОРОТ ПО QUATERNION
else{
vPosition=vRotated;
vec3 vcV=cross(quaternion.xyz,vRotated);
vPosition=vcV*(2.0*quaternion.w)+(cross(quaternion.xyz,vcV)*2.0+vRotated);
}


gl_Position=projectionMatrix*modelViewMatrix*vec4(vPosition+offset,1.0);


}


`;


fs["sprite"]=`


const int count=8;
uniform sampler2D map[count];
varying vec2 vUv;
varying vec4 vColor;
varying float vBlend;
varying float num;


void main(){


if(num==0.0){
gl_FragColor=texture2D(map[0],vUv)*vColor;
}
else if(num==1.0){
gl_FragColor=texture2D(map[1],vUv)*vColor;
}
else if(num==2.0){
gl_FragColor=texture2D(map[2],vUv)*vColor;
}
else if(num==3.0){
gl_FragColor=texture2D(map[3],vUv)*vColor;
}
else if(num==4.0){
gl_FragColor=texture2D(map[4],vUv)*vColor;
}
else if(num==5.0){
gl_FragColor=texture2D(map[5],vUv)*vColor;
}
else if(num==6.0){
gl_FragColor=texture2D(map[6],vUv)*vColor;
}
else if(num==7.0){
gl_FragColor=texture2D(map[7],vUv)*vColor;
}


gl_FragColor.rgb*=gl_FragColor.a; // ДЛЯ ПРАВИЛЬНОГО ОТОБРАЖЕНИЯ
gl_FragColor.a*=vBlend; // ЧЕМ МЕНЬШЕ, ТЕМ БОЛЬШЕ ADDITIVE. ЧЕМ ВЫШЕ, ТЕМ ГУЩЕ


}


`;

Grass InstancedBufferGeometry not rotation to camera

// ____________________ GRASS_LONG ____________________


OBJLoader.load("models/grass_long.obj",function(object){
mesh["grass_long"]=object.children[0];
setTimeout("instance_set();",4000);
instance_long();
});


function instance_long(){


var bufferGeometry=mesh["grass_long"].geometry;
var geometry=new THREE.InstancedBufferGeometry();
geometry.index=bufferGeometry.index;
geometry.attributes.position=bufferGeometry.attributes.position;
geometry.attributes.uv=bufferGeometry.attributes.uv;
geometry.attributes.normal=bufferGeometry.attributes.normal;


geometry.setAttribute('offset',new THREE.InstancedBufferAttribute(new Float32Array(),3));
geometry.setAttribute('orientation',new THREE.InstancedBufferAttribute(new Float32Array(),4));
geometry.setAttribute('scale',new THREE.InstancedBufferAttribute(new Float32Array(),1));
geometry.setAttribute('color',new THREE.InstancedBufferAttribute(new Float32Array(),3));


mat["grass_long_1"]=new THREE.ShaderMaterial({
uniforms:{
map:{value:tex["grass_long_1"]},
alphaMap:{value:tex["grass_long_1_a"]},
alphaMap2:{value:tex["grass_long_1_a_2"]},
noise:{value:tex["terrain_noise"]},
shadowMap:{value:tex["terrainCompleteMap"]},
time:{value:0},
sun_pos:{value:sun.position},
},
vertexShader:vs["grass"],
fragmentShader:fs["grass"],
side:THREE.DoubleSide,
});


mesh["instance_grass_long"]=new THREE.Mesh(geometry,mat["grass_long_1"]);
mesh["instance_grass_long"].matrixAutoUpdate=false;
mesh["instance_grass_long"].updateMatrixWorld=function(){};
mesh["instance_grass_long"].frustumCulled=false;
mesh["instance_grass_long"].onBeforeRender=function(){ this.material.uniforms.time.value=time*0.002 };
scene.add(mesh["instance_grass_long"]);
// ЗНАЧЕНИЕ ДЛЯ ОТСЕЧЕНИЯ АЛЬФАМАСКИ ДОЛЖНО БЫТЬ ТАКОЕ, ЧТОБЫ ПРИ НАБЛЮДЕНИИ НА ПЕРПЕНДИКУЛЯРНУЮ ПЛОСКОСТЬ НЕ БЫЛО БЕЛОЙ ПОЛОСЫ,
// КОГДА ТРЕУГОЛЬНИК ПОЧТИ ПОЛНОСТЬЮ СУЖЕН И РАВЕН 1 ПИКСЕЛЮ


vs["grass"]=`


attribute float scale;
attribute vec3 offset;
attribute vec4 orientation;
attribute vec3 color;
varying vec2 vUv;
varying vec3 vColor;
uniform float time;
uniform vec3 sun_pos;
varying float vLight;
varying vec2 shadowUv;
varying vec2 noise_uv_y;
varying float m;


void main(){


float dist=distance(cameraPosition.xz,offset.xz);
if(dist>100.0){ gl_Position=vec4(0,0,-1,0); return; }


vec3 vcV=cross(orientation.xyz,position);
vec3 vPosition=vcV*(2.0*orientation.w)+cross(orientation.xyz,vcV)*2.0+position;


if(position.y>0.0){
//vPosition.x+=sin(time*0.75+(vPosition.x+vPosition.y)*0.04)*vPosition.y*0.17+sin(vPosition.z);
//vPosition.x+=sin(time*0.75+(vPosition.x+vPosition.y)*0.04)*vPosition.y*0.1
//vPosition.z+=cos(time*0.5+(vPosition.x*0.5+vPosition.y)*0.04)*vPosition.y*0.1
//vPosition.x+=sin(time*0.75+cos(length(offset))+(vPosition.x+vPosition.y)*0.04)*vPosition.y*0.1*cos(length(offset));
//vPosition.z+=cos(time*0.5+cos(length(offset))+(vPosition.x*0.5+vPosition.y)*0.04)*vPosition.y*0.1*sin(length(offset));
vPosition.x+=sin(time*0.75+cos(length(offset)))*0.2;
vPosition.z+=cos(time*0.5+cos(length(offset)))*0.2;
}


// ТОЛЬКО СЕЙЧАС СТАВИТСЯ МАСШТАБ, ЧТОБЫ УВЕЛИЧИВАЯСЬ ПРИ ПРИБЛИЖЕНИИ, НЕ ДЁРГАЛОСЬ ОТ ВЕТРА


if(dist<95.0){ vPosition*=scale; }
else{ vPosition*=scale*((100.0-dist)/5.0); }


//vPosition.xz/=2.0;
vPosition*=1.0;
vPosition.y/=2.0;
vPosition+=offset;


noise_uv_y=vec2((position.x+offset.x)*0.002,(position.z+offset.z)*0.002*-1.0)+0.5;
shadowUv=vec2(vPosition.x*0.002,vPosition.z*0.002*-1.0)+0.5;


vUv=uv;
//vColor=color;
vColor=vec3(1.0+sin(vPosition.x)/5.0,1.0+sin(vPosition.y)/10.0,sin(vPosition.z)/10.0);
vLight=dot(normal,normalize(sun_pos));
if(vLight<0.2){ vLight=0.2; }


if(dist<2.0){ m=0.0; }
else{ m=(dist-2.0)/20.0; }


gl_Position=projectionMatrix*modelViewMatrix*vec4(vPosition,1.0);


}


`;


fs["grass"]=`


uniform sampler2D map;
uniform sampler2D alphaMap;
uniform sampler2D alphaMap2;
uniform sampler2D noise;
uniform sampler2D shadowMap;
varying vec2 vUv;
varying vec3 vColor;
varying float vLight;
varying vec2 shadowUv;
varying vec2 noise_uv_y;
varying float m;


void main() {


vec3 diffuse=texture2D(map,vUv).rgb;
float a=mix(texture2D(alphaMap,vUv).r,texture2D(alphaMap2,vUv).r,m);
if(a<0.5){ discard; }
//if(texture2D(alphaMap,vUv).r<0.5){ discard; }
diffuse*=vColor;
diffuse*=texture2D(noise,noise_uv_y).rgb;
diffuse*=texture2D(shadowMap,shadowUv).rgb;
//diffuse*=clamp((vUv.y+0.25)*3.0,0.0,1.5);
/*
float depth=gl_FragCoord.z/gl_FragCoord.w;
float fogFactor=smoothstep(10.0,20.0,depth);
diffuse=mix(diffuse,vec3(0.55,0.7,0.77),fogFactor);
*/
gl_FragColor=vec4(diffuse,1.0);


}


`;

InstancedMesh of PlaneGeometry and MeshBasicMaterial as sprites: Mesh points to the camera on only 2 axis with shaders - #7 by prisoner849

I made a billboard based on your sources.
But as you move away, you suddenly lose sight of it.

What’s the problem?

https://jsfiddle.net/avatar_studio/vxy09m3p/12/

I applied the code you gave and tested it.
It was very simple.
However, when looking at the ground from the sky, the image is not visible.
Of course, it is an accurate movement, but when looking at the ground from the sky, the terrain is natural when you can see about 2-30% of the vegetation.
I tried changing the numbers to try, but it was difficult.
How do I do that?

https://jsfiddle.net/avatar_studio/nb4qwp3g/1/

Hello. Found that when instancedMesh goes into a child object and rotates, billboard throws an error.
How can I correct the error?

https://jsfiddle.net/avatar_studio/nb4qwp3g/8/

I’ve got no error messages in the console :thinking:

There are no error messages. If you look at the execution screen, you can see that the billboard does not work.

The trees should always be visible, but the trees disappear.

Ah, got it :slight_smile: Will have a look later

Use side:THREE.DoubleSide,
image

It seems completely invisible at 90 degrees.
It definitely looks like the code needs to be fixed.