Video Texture updates sometimes showing previous frame

Questions
1

In this test example (make sure to inspect source) I try to set the currentTime to some frame before showing the texture. Ie. I don’t ever want to show frame 0. In this particular video, the correct behavior would be to immediately load to the frame that appears half cut-off. This works fine in Chrome, but in Edge, Safari, and sometimes Firefox, it somehow forces me to render frame 0. Try tapping the screen and it will force a needsUpdate and then show the new frame, but no matter how long those setTimeouts are, it always will show frame 0 initially. Can anyone see why?

2

I suggest trying to wait until the onloadeddata or onloadedmetadata before you start to try and render your video

video = document.createElement( 'video' );

video.width = 640;
video.height = 360;
video.muted = true;
video.autoplay = true;
video.src = 'https://files.panomoments.com/media/example.mp4';
video.crossOrigin = 'anonymous';
video.setAttribute( 'webkit-playsinline', 'webkit-playsinline' );

video.onloadeddata = funcction(){
	video.pause();

	texture = new THREE.Texture( video );
	texture.minFilter = THREE.LinearFilter;
	texture.format = THREE.RGBFormat;
	material   = new THREE.MeshBasicMaterial( { map : texture } );
	mesh = new THREE.Mesh( geometry, material );
	mesh.rotation.y = Math.PI;
	scene.add( mesh );
}
3

Thanks for the suggestion, but that doesn’t seem to change the behavior on either Edge or Safari. It still wants to render that first frame. It’s almost like there’s a queue of textures…

4

FYI, this also still happens if I first download the entire mp4 and load it as a local Blob. So I don’t believe it’s an issue inside the video download / streaming code of the browsers.

5

Here’s an example that is closer to how I’d actually like to use this code (downloading as Blob and doesn’t
rely on those test setTimeouts) - https://files.panomoments.com/js/videoSeekSafari.html

With this design, I am back to only having issues on Safari (which is why I originally posted as a bug here) - Any ideas on why this is still misbehaving?

6

Update - And even when further waiting another 5 seconds after the seek completes before creating the texture, Safari still shows that 0 frame:

			if (!videoInitialized && video.readyState == 4 ) {
				console.log("videoInitialized")
				video.pause();
				video.currentTime = 40;
				videoInitialized = true;
			} else if (!videoSeeked && videoInitialized && video.readyState == 4 ) {
				console.log("videoSeeked")
				videoSeeked = true;

				setTimeout(() => {
					console.log("setTimeout")
					texture = new THREE.Texture( video );
					texture.minFilter = THREE.LinearFilter;
					texture.format = THREE.RGBFormat;
					material   = new THREE.MeshBasicMaterial( { map : texture } );
					mesh = new THREE.Mesh( geometry, material );
					mesh.rotation.y = Math.PI;
					scene.add( mesh );
					texture.needsUpdate = true;
	            }, 5000);

			}
7

Can you create an online codepen/fiddle that clearly shows the problem?

I’m unsure of what the expected result should look like, and what the problem result you are getting is. On Edge I get black screen until it loads the panorama, in Chrome I get black screen until it loads a panorama with a black circle in the middle…

Maybe make your video a colour gradient with red as the first frame and blue as your target frame, that way you can see ‘I want the thing to be blue without ever showing the red’ so it is clear for people unfamiliar with your media to help?

1 Like
8

Here is a JSFiddle with a color coded example - https://jsfiddle.net/rj7L0rsa/

Red is the target color. Light blue/green is the first frame which should never be shown. Note how there is a setTimeout in the second condition - No matter what value this timeout is set to, it will always show the incorrect frame on Safari.

Note - For some reason this JSFiddle doesn’t run properly in Firefox. Just ignore that as I’m only really interested in the Safari issue. Also, when you tap the mouse after waiting for the timeout, it should show the correct frame. However, there is another unrelated Safari Webkit issue that causes a black flash, so you may need to click a few times to see the red frame. If you download the current Safari Technical Preview this black frame issue is fixed.

9

Thanks for that, very interesting problem… The best I could do to solve it is: https://jsfiddle.net/rj7L0rsa/40/

What I’m doing is loading in the mesh and video with 0.001 opacity, and then changing it to 1 opacity after a setTimeout. Not a great solution, but the only one I could get working at this point.

i believe there is something happening between compile time of the shader and loading of the video. The shaders in ThreeJS only compile once they are in view of the camera. Safari may have an optimisation to not begin loading the video into the GPU until its actually being used. The setTimeout of 1000ms is used (100ms too short on my device) to wait for the compile time and load time of the shader.

Soooo I’m pretty stumped as to making a proper solution…

10

Yeah, it’s an strange one :slight_smile: Thanks for taking a look. I may be able to use that as a workaround.