ThreeJS for Seamless Video Player: Not Seamless


I’m brand new to ThreeJS and to the forum here. I’ve been running some test to see if ThreeJS will be useful in setting up a web-based seamless video player. The results so far have been really impressive, but I’ve run into a snag that I simply have not been able to get my head around.

ATTEMPT: I’m trying to programmatically switch between two video sources. Right now, the method basically initiates the two videos what will need to swap as two separate plane meshes in a scene with different z positions on an orthographic camera (IE, one behind the other). The first video plays, the second (not visible) video stays paused until called upon then at some later, pre-defined point, let’s say 5 seconds for the sake of this example, then the program calls for the two videos to change z-positions and immediately calles .play() method on the second (paused) video.

This words fine in basic HTML 5 with video tags and CSS, but when I introduce Three, it introduces a stutter on the “play” method of the second video. The first maybe 6 or 8 frames are dropped, and then the video catches up to where it should have been.

This happens particularly in Chrome, but Safari has it’s own issues that I won’t get into because that’s not really in my ecosystem

One thing I’ve tried with some success is to trigger a few random and unseen play/pause actions behind the scenes, but the timing here is really tricky. For one video if I pull this toggle off 500ms before I actually want to SEE the video, it works great, for another 500ms is way to quick and it needs to be 1500ms before the video actually needs to be used.

I assume there’s some kind of resource management happening here? If anyone can help, I’d very much appreciate it. Open to other methods as well if there are suggestions, but any approach here needs to understand that the video frames are going to have to appear seamlessly-- the last frame of the front video has to appear continuous with the first frame of the rear video. At 60fps, that’s about 16.67ms execution time or better, which I don’t think would be requiring too much.

Very much appreciate the insights.

This is how the videos are initialized.

async function initVideo(videoPath, autoPlay, zIndex, opacity) {
  return new Promise((resolve) => {
    const video = document.createElement("video");
    video.src = videoPath;
    video.crossOrigin = "anonymous";
    video.loop = true;
    video.muted = true;
    video.preload = "auto";
    video.playsInline = true;

    if (autoPlay) {
      video.autoplay = true;

    const opacityValue = opacity || 1.0;

    const canPlayThrough = () => {
      const videoTexture = new THREE.VideoTexture(video);

      const customShader = {
        uniforms: {
          tDiffuse: { value: videoTexture },
          contrast: { value: 1 },
          saturation: { value: 1 },
          brightness: { value: 1 },
          gamma: { value: 1 },
          colorTemperature: { value: new THREE.Vector3(1, 1, 1) },
          opacity: { value: opacityValue || 1.0 }
        vertexShader: glslVertexShader,
        fragmentShader: glslFragmentShader,

      var shaderMaterial = new THREE.ShaderMaterial({
        uniforms: customShader.uniforms,
        vertexShader: customShader.vertexShader,
        fragmentShader: customShader.fragmentShader,
        transparent: true,

      var planeMesh = new THREE.Mesh(planeGeometry, shaderMaterial);

      planeMesh.position.set(0, 0, 0);
      planeMesh.position.z = zIndex;
      planeMesh.rotation.y = THREE.MathUtils.degToRad(parseFloat(0));
      planeMesh.rotation.x = THREE.MathUtils.degToRad(parseFloat(0));
      planeMesh.rotation.z = THREE.MathUtils.degToRad(parseFloat(0));
      planeMesh.position.set(parseFloat(0), parseFloat(0), parseFloat(zIndex));
      const videoId = createVideoId();
      const videoItem = new VideoItem({
        name: videoPath,
        filePath: videoPath,
        domElement: video,
        videoTexture: videoTexture,
        videoMaterial: shaderMaterial,
        planeMesh: planeMesh,
        id: videoId,

      videos[videoId] = videoItem;

      console.log("loaded videos: " + loadedVideos.length, loadedVideos);


    video.addEventListener("canplaythrough", canPlayThrough, { once: true });

    if (autoPlay) {;

And this is the function that swaps the videos…

async function swapBuffer() {
  const frontVideo = nowPlaying;
  const frontVideoIndex = playQueue.findIndex(
    (item) => item.videoId === frontVideo
  let backIndex;
  frontVideoIndex === 0 ? (backIndex = 1) : (backIndex = 0);
  const backVideo = playQueue[backIndex].videoId;
  videos[frontVideo].planeMesh.position.z = -0.01;
  videos[backVideo].planeMesh.position.z = 0;
  nowPlaying = backVideo;
  console.log("nowPlaying:", nowPlaying);

what if instead of VideoTexture you use Texture, and manually set texture.needsUpdate = true every frame?

1 Like

Thanks for the reply and the suggestion. I’m not certain in that setup how I would parse the video source. The html video tag doesn’t update by frame and doesn’t know the framerate. CurrentTime is useful with requestanimationframe, but I’d still have to somehow decipher the video’s framerate in a browser environment.

yeah yeah you can save that for later, for now just constantly update and lmk what happens :laughing:

Hey, makc3d.

Looks like the update to Texture (versus VideoTexture) and the needsupdate on each requestanimationframe worked for the test. Cut is smooth now. Any idea what’s causing this? I’m not familiar enough with Three to be able to guess at what I’m losing here by using Texture versus VideoTexture, but I would guess a bit of optimization and maybe access to some video render effects?

Thanks for the help.

VideoTexture is using video.requestVideoFrameCallback or video.HAVE_CURRENT_DATA to guess when the texture needs to be updated - obviously it guesses wrong in your case :person_shrugging:

I suppose you could skip every other frame since normal video frame rates are below 30 so there is no point in updating at 60 fps

1 Like

Thanks. You know of any other differences between the two methods? I use glsl shaders and was looking into some of Three’s post-processing effects.

Weird thing is that it was guessing right in Safari. I’m guessing it’s a Chrome resource management thing that affects the way the video tag is handled if Chrome knows the changes are not rendering to visible dom.

no difference on glsl side