Hello! So this is driving me nuts and I suspect this is just a limitation of browsers at the moment but I’m hopeful. When using something like the mouse move event to track the mouse cursor and using requestAnimationFrame for rendering you get a very perceptible input lag of 1 frame between the last available mouse position and rendering. This affects raycasting, objects attached to the cursor, camera movement, etc. You can see an example of the lag here (enable the software cursor toggle).
It seems that the input events all get fired after the rAF call which in the common case makes sense – rAF should fire at the beginning of the frame to afford the most amount of time to do work before the page displays. But in practice I’d like to perform my rendering as close to the page update as possible so I have the most up to date input state.
One option is to use on demand rendering as some of the three.js examples do and beyond the obvious limitation of not being able to render animations every frame unless the user triggers an event, multiple events can run per frame (and in Safari events like move move get called up to 8 times between rAF). This leads to rendering the scene multiple times per frame even if it’s debounced using something like Promise.resolve() or queueMicrotask.
So my questions are: has anyone dealt with this successfully? Is there anything in modern browsers to handle this? Is there a way to get a callback after at least some mouse events have fired? Or some way to use both the on demand rendering approach and the rAF approach while still only rendering once per frame? Ideally I’d want a callback that lets me fire it X milliseconds before the end of a frame so I have more up to date data. Or some global call that lets me get the current hardware cursor position without having to wait for a mouse event but I know those don’t exist (yet? ).
@usnul thanks for the suggestions! Can you elaborate on how you imagine using the event options here? From what I can tell “passive” is exclusively to help smoothen scrolling and “capture” changes the order in which events bubble up the dom tree so it doesn’t seem like they’ll help with event ordering relative to rAF.
Using an OffscreenCanvas is an interesting idea. I’ll have to learn more about it and think through how that will work with window events. Unfortunately support for it seems pretty poor and passing all scene changes through the worker postmessage API for complex apps seems like a non starter. At that point I might just have to deal with the 16ms latency
for offscreen canvas. I suggest you can just keep all your geometry and materials in that thread, and exchange messages with the assumption that the worker thread is the owner of graphics primitives. It will be much easier than you think (probably), but of course that’s assuming that you’re starting with the greenfield, if you have a fair amount of existing infrastructure in place - it will be a royal pain for sure.
For the event listener parameters. I mean try every permutation and see if it has any effect. I’m guessing that depending on the flags the internal implementation might dispatch events sooner. That’s just a hunch, and if it turns out to be true - it will heavily depend on the implementation in a specific browser.
Just a bit more thoughts on the subject:
16ms is not so bad, and screens tend towards higher refresh rate as of late, I have a 144Hz display myself, because when I was making an upgrade it was about the same price as my old 60Hz monitor that I was replacing. With that you’re in 7ms domain, and it will probably only improve from there.
I agree that this is no optimal, for something like an FPS or a fighting game it’s really bad, but it’s generally okay for non-fast paced real-time application (games?). Perhaps we will get a better API for low level Input handing as time goes on.
Yeah part of me was hoping there was some combination of well defined modern browser features to help with this. With all the other functions that have been added to afford more control over when callbacks get fired I’m surprised there’s still nothing more that can be done here.
And of course 16ms isn’t disastrous but it’s very noticeable and feels like a performance problem in some cases. And really it’s at least 16ms delayed since that’s just the CPU delay – there’s still additional delay from having to wait for the GPU to finish drawing in order to display that already stale mouse position. Common 144Hz monitors still seems like a long ways out and my users are all on standard 60Hz displays. Even if 144Hz was an option that brings a new host of issues including having to suddenly squeeze all of your processing per frame into 7ms rather than 16ms.
Perhaps we will get a better API for low level Input handing as time goes on.
Hoping for this! I wonder if there’s anything already in the pipe.
function animate() {
requestPostAnimationFrame(() => {
// Get input here
requestAnimationFrame(() => {
render();
animate();
});
});
}
I don’t know if this will ever be added, however, I think it’s possible to create the same behavior by wrapping requestAnimationFrame in a promise that resolves immediately. Or, there’s a polyfill here: afterframe.
See also this blog post which goes into more detail:
Thanks @looeee these links are really helpful. I see I was misunderstanding when mouse events were being fired – specifically that they’re fired exactly before rAF just at the beginning of the frame so the visible delay should just be the movement difference since the beginning of the frame.
I dig through these a bit more and see what else I can glean. Thanks again!
).