Hey cool peeps! Just wanted to introduce myself and share some of the projects I’ve been working on! I’m actually new to programming and at this point all I really know was learned while working on this PortalCamera project over the past year.
It was a long process of researching, trial and error and too many prompts than i’m willing to admit but I’ve finally got the tools to a place that they’re pretty performant and are a good foundation to dive deeper and make games or portal experiences with.
There’s actually been quite a few versions of this type of rendering over the years but most of them are outdated or required hardware that I didn’t have. The best solution I could come up with was using a vision model from mediapipe to track the user’s “gaze-point” between the eyes and create a coordinate output that was relative to the user’s monitor.
From there I had to adapt Robert Kooima’s General Perspective Projection formulas to a webgl format that three.js could utilize.
there’s an auto-calibrate wizard to get an initial alignment to your monitor and from there you can change the gains and offsets to better line things up.
This version isn’t as fast as the standalone version where I separate the tracker from the renderer.
You can run this head tracker as its own web application and it connects via a websocket to this standalone portalcamera demo
and the last link (i promise) is basically another standalone demo that is as minimal as possible, it’s a great way to really see the core code that connects all the different components: minimal portal camera demo.
I apologize if this is confusing, there’s more technical details in the repos but I’m just excited to share this project, and see if anyone else is excited about it too .
Let me know if any of you end up trying it out and how it works or if you have issues!
Even without the head tracking element, being able to skew it could be useful for literal projector maps like luma map or the light form projectors. I always found that stuff so cool and still get a kick out of those Madison square garden ads that mess with perspective.
.