Video on a texture very slow on devices

Hi there,

I can’t provide a jsfiddle because I won’t be able to overcome the cross origin problem when it comes to render a video on a texture.

The thing is, that my example is very very slow. I want to have a 360º video rendering on a scene, and an OrbitCamera inside it so you can look like a panorama using your mouse, touch or gyroscope.

Here is some of my code:

this.container = document.getElementById( 'containerVideo360' );

this.camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 1, 1100 );
this.camera.target = new THREE.Vector3( 0, 0, 0 );
this.camera.position.z = 0.001;
this.camera.target = new THREE.Vector3( 0, 0, 0 );
this.controls = new THREE.OrbitControls( this.camera, this.container );

this.scene = new THREE.Scene();
this.geometry = new THREE.SphereGeometry(64, 32, 32);
this.geometry.scale( - 1, 1, 1 );

this.videoDomElement = document.getElementById('myVideo');
this.videoDomElement.setAttribute( 'webkit-playsinline', 'webkit-playsinline' );
this.videoDomElement.setAttribute('crossorigin', 'anonymous');

this.texture = new THREE.Texture( this.videoDomElement );
this.texture.minFilter = THREE.LinearFilter;
this.texture.format = THREE.RGBFormat;
this.texture.generateMipmaps = false;

this.material   = new THREE.MeshBasicMaterial( { map : this.texture } );

this.pano = new THREE.Mesh( this.geometry , this.material);
this.scene.add(this.pano);

this.light = new THREE.AmbientLight( 0x404040 );
this.scene.add( this.light );

this.renderer = new THREE.WebGLRenderer(); //{antialias: true , alpha:true }
this.renderer.setClearColor( 0x000000, 0 );
this.renderer.setPixelRatio( window.devicePixelRatio );
this.renderer.setSize( window.innerWidth/2 , window.innerHeight/2 );

this.container.appendChild( this.renderer.domElement );

animate();
...................................

animate() {
    update();
    requestAnimationFrame( this.animate );
}

update() {
    this.texture.needsUpdate = true;
    this.renderer.render( this.scene, this.camera );

    if( this.props.isMobile ) {
      this.controls.update();
    }
  }

The problem is that it runs very slow. My video has 1024x512 size and the bitrate is not crazy, just around 6000kbps.

Any ideas why this could be running so slow on mobile devices?

Thanks a lot…

Ricardo

With how many mobile devices have you actually tested?

Do you have performance problems with examples like
three.js webgl - equirectangular video panorama

It runs smoothly on my Pixel.

I have tried with a Xiaomi MI5 and Iphone7. It should run ok on these devices.

The example you mention runs smoothly on my testing devices too… But I don’t see any big difference, except:

  • The video quality is very poor, but the resolution is around 2K…
  • It is not using the orbitControls… just handling that manually…

You might want to change your code to something like:

setInterval( function () {

	if ( video.readyState >= video.HAVE_CURRENT_DATA ) {

		texture.needsUpdate = true;

	}

}, 1000 / 24 );

In this case, you don’t update your video texture each frame. Read the following issue for more information:

1 Like

I don’t notice any improvements when I try with an interval instead of RAF.

It must be something else…

I’ll try removing orbitcontrols to see if there is an overload because of that…

Alright, I found out what the problem is.

I realized that the performance impact is happening when we use gyroscope. It runs ok on mobile devices, when I disable the gyroscope or have my mobile phone laying flat on the table.

I can then touch the screen and it runs smoothly, so clearly the problem here is the gyroscope generating a lot of extra rendering or something like that.

I don’t really know what to do now… I wouldn’t like to disable it.

Any ideas @Mugen87 ?

Thanks so much!

So you are using THREE.DeviceOrientationControls? Or how do you process orientationchange and deviceorientation events?

I added some code to OrbitControls so it could handle it.

At the end of the update method (OrbitControls):

// Gyroscope Additions
			if ('undefined' === typeof scope.deviceOrientation) {
				return false;
			}
			
			var alpha = scope.deviceOrientation.alpha ? THREE.Math.degToRad(scope.deviceOrientation.alpha) : 0;
			var beta = scope.deviceOrientation.beta ? THREE.Math.degToRad(scope.deviceOrientation.beta) : 0;
			var gamma = scope.deviceOrientation.gamma ? THREE.Math.degToRad(scope.deviceOrientation.gamma) : 0;
			var orient = scope.screenOrientation ? THREE.Math.degToRad(scope.screenOrientation) : 0;

			var currentQ = new THREE.Quaternion().copy(scope.object.quaternion);
	 
			setObjectQuaternion(currentQ, alpha, beta, gamma, orient);
			var currentAngle = Quat2Angle(currentQ.x, currentQ.y, currentQ.z, currentQ.w);
			var radDeg = 180 / Math.PI;
	 
			rotateLeft(lastGamma - currentAngle.z);
			lastGamma = currentAngle.z;
	 
			rotateUp(lastBeta - currentAngle.y);
			lastBeta = currentAngle.y;
	 
			return false;

I just realized that, yeah, pretty much, my code was taken from DeviceOrientationControls so I guess there shouldn’t be anything weird there… I’ll check

Where are you calling this.texture.needsUpdate = true;?

Oh man…

I’m calling needsUpdate inside the update function (first comment above).

BUT, the problem was… [drumroll]… that I’m using a librarly called ViewportUnitsBuggyfill, which aims to create a consistent behaviour between browsers when it comes to VH and VW… and I had set up this event handler somewhere in my code:

window.ondeviceorientation = function() {
      viewportUnitsBuggyfill.refresh();
    };

and, what I needed was:

window.onorientationchange = function() {
      viewportUnitsBuggyfill.refresh();
    };

So, pretty much, on every tiny change of orientation, I was triggering a refresh of EVERY vh and vw in my big website, which is madness.

I apologize for the inconvenience. My mistake…

Thanks a lot for all the help that you provide

By the way… I will open a different post if think I have to, but I wanted to ask something else…

On a different part of my web, I’m using a Three.VideoTexture in order to render some videos inside my sphere.

The problem is, that eventually they start showing glitches… like green and white spots, back and forth frames quickly happening…

I guess it is related with this conversation that your already posted:

Should I not use VideoTexture and manage my updating through an interval instead? would that solve my problem?

Thanks a lot…

Sounds like a video decoder issue. You should report that to the browser directly.