This would be identical to CSS perspective-origin.
For now, I’ll come up with something using camera frustum size and WebGLRenderer.setViewport.
Having the API built into perspective cameras would be nice.
Is it the same concept as the one with THREE.StereoCamera? It creates two cameras, for both eyes, and each camera is shifted and the projection is not symmetrical. So, it is a kind of changed perspective origin, I think.
See lines 57-89:
Ah yeah, that may be a hint for the math way of doing it instead of with setViewport. Thanks! Need to figure how to apply that based on a distance from the edge of the viewport.
I’m finally looking into this.
Sidenote, I think the “off-axis” approach seems not accurate enough for VR if we consider that in a VR headset our eyes can change rotation to look at different parts of the screen. I’m thinking that the best approach, in a headset that can support it, may be to dynamically change the perspective origin to match where the user’s eyes are focused. But it’s just a theory.
Perspective origin is useful for non-VR non-stereoscopic cases too though. For example, a simple use case (that occurs often) is to shift the center/origin of a scene over so that a model being viewed (f.e. rotating a camera around the model with OrbitControls) will be centered in a area not covered by a UI panel.
Here’s an example using only CSS (no Threejs) to illustrate a shift perspective-origin and the scene’s origin to the right so that 3D content is centered in the area not covered by a UI panel:
Video:
To better illustrate, here are screenshots without the content having rotated.
Here’s what it would look like without a shift in perspective-origin (comment out the line with perspective-origin) and only a shift of the origin. Content is located at X=0 without any rotation, the cube faces are perpendicular/parallel to the origin axes, but the cube is not visually symmetrical because the origin shift does not also include a perspective shift, and it is as if the content was actually translated (not visually symmetrical):
Now here is a screenshot with the perspective-origin restored and moved along X to the center of the area not covered by the UI panel, such that the cube is visually symmetrical around the Y axies (X=0) of the scene:
Note how the content does not appear to be skewed rightward in the second screenshot, as it was in the first screenshot when I simply moved the content over. The whole gray area under the UI is the 3D scene, and as the scene resizes, the content remains symmetrical while not in the middle of the screen. The center/origin of the 3D scene is translated to the right, and anything centered there remains visually symmetrical on the screen. It is as if the right edge of the 3D scene is offscreen and clipped outside of the window (hence why mentioned in the OP one way to do this is with WebGLRenderer.setViewport, to clip the camera window on one side).
I’ll implement this as a perspective-origin HTML attribute in Lume’s 3D elements (built on Threejs), but I think it would be neat to incorporate this in Threejs directly somehow (seems like it would make sense as a property of PerspectiveCamera). If not in Threejs directly, I may publish a function for plain Threejs that can modify a camera’s perspective transform.
1 Like
Would it be possible to achieve the effect of perspective-control lenses to reduce perspective distortion?
That would be interesting! I always wondered about this for wide aspect scenes, where objects near the edge are so distorted. I’m more wondering if the scene could be spherically distored (the cornea is somewhat spherical), instead of rectangular. But its a different (though related) topic to the perspective shift.
1 Like
Hm. I’m not sure spherical distortion can be completely eliminated or created with a single transformation matrix, as matrices preserve straightness of lines – i.e. a straight line cannot be projected into a curved line and vice versa (except in some very rare cases). For this you’d need either multiple matrices or a curved projection “plane” or some non-matrix (post-)processing.
2 Likes
Alright I tried a first version in Threejs. It was a lot easier than I thought!
Although WebGLRenderer.setViewport works mathematically, we actually want PerspectiveCamera.setViewOffset because renderer.setViewport clips the result while camera.setViewOffset affects only the projection matrix so it does not clip the result.
This example has a similar alignment effect as the CSS example:
Video:
I’m missing something small in both examples though. In both exampes, the content is not perfectly centered, as we can see when window is narrow enough:
In those two shots, the panel edge is touching the content, but the content is not touching the right edge of the window.
What is missing?
The CSS does not used --ui-panel-gap. Maybe the ui class should have:
left: var(--ui-panel-gap);
In the JS code, on my machine, I had to shift 10 pixels. Tried with different panel sizes and gaps, it is always 10 pixels. Don’t have the time to look for where did it come from. Some border? The width of the hidden scrollbar?
camera.setViewOffset(..., -10 - (uiPanelWidth+uiPanelGap)/2, ...)
Isn’t the barrel distortion shader what handles all of this? It simulates the scene projected on a sphere on each eyeball… so where your eyes are pointing doesn’t matter. It also includes an inverse chromatic abberation to compensate for the refractive effect created by the lenses… And none of this is something that can be captured by CSS transforms unless its running inside a VR enabled browser.
Maybe I’m not understanding what is being discussed here. 
I’m also not completely sure, but most likely it is this: sometimes the center of the canvas is not the visual center of the active area, because there might be some HUD panels, menus, etc. I have experienced this issue in Platons. The platon is drawn in the middle of the canvas. When you reszie the window, it is still in the middle of the canvas, but this is good. It should be in the middle of the empty space of the canvas. So @trusktr is trying to modify the projection matrix to fix this.
Ahhh I see. Yeah… that would require some kind of customized projectionMatrix.
I encountered this a few times… and the first time, I “fixed” it by not allowing the UI to overlap the canvas… and the second time, by just changing the orbitcontrols target to aim off center of the object by some fixed X in camera space. I’ve never tried to compute a modified projectionMatrix. I wonder how well that will play with frustum calculations? I’d think they assume some kind of symmetry in the frustum axes… but maybe this isn’t an issue?
2 Likes
Oh, I think you loaded it before I was done. It already has that.
The topic is actually much simpler (barrel distortion ideas were off-topic). I’m simply translating the origin of the scene so that it is not centered, just like we can with CSS perspective-origin along with a CSS translation of the content.
Barrel distortion of course cannot happen in CSS because we cannot apply arbitrary transforms to the camera, we can only apply perspective and perspective-origin to the CSS camera and that’s about it, but that’s another topic.
So, the goal was to reproduce the CSS perspective origin shift in Threejs (first demo is CSS, second demo is Threejs) and it turned out to be super simple! The demos translate the origin/perspective so that the origin of the scene is centered in the area not covered by a floating UI panel.
Turns out it only requires a call to camera.setViewOffset (which indeed affects the projection matrix underneath).
That’s simple, although in my case I’m going for UI floating on top. I updated the above examples to have a blurred glass UI:
Got an example?
I didn’t this time either. camera.setViewOffset did all the work.
camera.setViewOffset() results in the projection matrix being modified, and frustum.setFromProjectionMatrix() defines the frustum from that matrix, so I’m guessing that it should take this into account (it should affect the top/bottom/left/right planes).
1 Like
Hmmm, where is this offset coming from, in both CSS and Threejs???
The CSS demo was scrolling vertically, so I disabled scroll with overflow: hidden, but the problem persists. Even with scrolling, in Chrome the scrollbar is overlaid on top of the content and it is transparent, so that wasn’t causing any layout shift anyway.
I’ll come back to this offset problem later.
Ok, now I wonder, how do we change the perspective origin without translating the content in Threejs, like in CSS 3D?
For example, here’s a fork of the previous CSS 3D demo where I’m animating only the perspective-origin:
Video:
Basically when we animation perspective-origin, the content stays in the same position in space, but only the perspective changes. It basically skews the whole scene so that wherever the perspective origin point is inside of the view, content at that location has perspective perpendicular to the screen (parallel with the line of site).
To achieve this we could use camera.setViewOffset(), which moves both the content and the perspective origin, and then translate the whole scene the same amount in the opposite direction (either translate the Scene, or translate the PerspectiveCamer itself) but then I believe this will break a bunch of other features like lookAt (f.e. calling camera.lookAt(target) will undo effects of these translations).
How can we achieve this without touching the transforms of any object in the tree (for example, without translating the root Scene, or the Camera), so that things like lookAt will work as expected? Could we apply this in the projection matrix somehow?