How to live stream animations of gltf and achieve frame rate independence

Hello there,

I have a gltf model and receive animation data from a server for that model. I periodically receive frames from the server. Each frame consists of morph Targets and bone transforms. My goal is to animate the gltf model dynamically as I receive the frames.

My biggest constraint will always be the latency to my server, but I am still wondering about the best ways to approach this problem.

My first approach was to receive a frame, apply the transforms and then call renderer.render(scene, camera), however the animation is not very smooth.

A better approach was using a render loop like so:

const clock = new THREE.Clock()
let previousTime = 0
const tick = () =>
{
    stats.begin()
    const elapsedTime = clock.getElapsedTime()
    const deltaTime = elapsedTime - previousTime
    previousTime = elapsedTime
    
    // Update controls
    controls.update()
    
    // Render
    let start = performance.now()
    renderer.render(scene, camera)
    let end = performance.now()
    
    window.requestAnimationFrame(tick)
    
    stats.end()
}
tick()

The animation is smooth in this way, but still it is not frame rate independent for different devices , as I am unsure of how I should use the deltaTime I calculate above. Usually I would multiply any movement (i.e. rotation, change in position) by this deltaTime in the render loop directly, but it is not clear to me how I can combine the animation loop with the constraint of receiving the frames from a server (I don’t know when I will receive the next frame)

The code below is triggered by the EventListener each time I receive a frame. How can I use deltaTime here when it is only calculated in my render loop? Does a render loop make sense when I am receiving the frames from the server like this?

    ws.addEventListener("message", AnimationData => {
        let {blendshapes, transformations, time} = parseRenderFrame(AnimationData);

        //Set the morph Targets
        animateMorphTargets(model, blendshapes)

        //Animate skeleton
        let bones = skeleton.bones;
        animateSkeleton(model, transformations, bones);
    })
}

Use setInterval().

I am doing that currently for sending the frames over. Can you elaborate on how I should use setInterval()?

To power the animation loop, one can use requestAnimationFrame(), setInterval() or setTimeout().

The RAF callback delay is set by OS screen repaint frequency. The SI and STO are set by you.