Or at least, a very small, budget-friendly version of it.
Long time lurker, first time poster here.
My actual question is fairly simple… after a normal render pass, how can I use a ShaderPass to take the pixels inside a quadrilateral and stretch/skew them to fill the full render frame? Edit: I think this is called “unprojecting.”
This quadrilateral is defined by four Uniform values expressed as x:[0.0 - 1.0], y:[0.0 - 1.0]. These are passed in from javascript.
Here’s the shader code I have. (With all my failed attempts removed)
CornerPinShader.js (1.4 KB)
I’ve worked at this for days, but I cannot grasp matrix math. These posts have helped me, but I’m still not there.
- Python script for Nuke (this does exactly what i need, but I’m stuck on “mapUnitSquareToQuad”)
- https://www.shadertoy.com/view/lsVSWW
- httpx://iquilezles.org/www/articles/ibilinear/ibilinear.htm
(Apologies for the “httpx,” but the forum limits links for new users)
Why do I want to do this?
It’s part of my attempt to re-create a budget version of “The Volume” that is starting to replace green screens in some VFX projects.
Original implementation of the idea over a decade ago. (This is closer to what I’m attempting):
My explanation below is kind of long, so here’s an image with the full story.
The end result is that in my real-world studio, we will create the illusion that our studio camera is looking “through” an LED wall at the 3D scene behind it.
The motion tracked position of the studio camera will drive the perspective of the 3D scene shown on the LED wall.
As the camera moves around the studio, the perspective of the 3d scene changes to create the illusion of depth for anyone standing near the camera. (It will look wonky for everyone else)
The ThreeJS part works like this:
I have a motion-tracked real-world camera in a TV studio. Its position is linked to the virtual camera in my ThreeJS scene.
There are four invisible Object3Ds in my virtual scene that correspond the four corners of a large LED wall in the real-world studio. This LED wall is the “window” through which we will view our 3D scene.
On each ThreeJS render frame, I calculate the screen positions of those points in a uV friendly range of x:[0…1] y:[0…1].
I do one normal render pass to get the full camera view of the 3D scene, then send those 4 corner points to a ShaderPass that will (this is where I am stuck) stretch/skew the pixels inside the “window” created by those four points so that they fill the full render frame.
So that’s my question… I’m stuck on the GLSL in the ShaderPass to stretch/skew those pixels to make them fill the full render frame.
There are issues I still need to resolve, such as matching camera FOV and lens distortion, but I will leave those for later.
Thanks for making it this far!!