How to get one PerspectiveCamera based on the `intrinsic` and `extrinsic` of a real camera?

Questions
1

This is a real camera parameter:

const originExtrinsic = [-0.9994466143126584, 0.033033376071303994, -0.003906559137689193, 0.20487898588180542, 0.0025198193977806005, -0.0419178508124942, -0.9991178830816032, 0.0013696063542738557, -0.033167991334523576, -0.9985748293686324, 0.04181141593201179, -0.10943480581045151, 0, 0, 0, 1]

const originIntrinsic = [1.2100629810000000e+03, 0.0, 1.0224299030000000e+03, 0.0,1.2058507139999999e+03, 7.9254164400000002e+02, 0.0, 0.0, 1.0]

the real camera photo size: width(2048px), height(1536px)

I transform them into internal parameter matrix and external parameter matrix

const extrinsic = new Matrix4()
extrinsic.set(originExtrinsic[0], originExtrinsic[1], originExtrinsic[2], originExtrinsic[3], originExtrinsic[4], originExtrinsic[5], originExtrinsic[6], originExtrinsic[7], originExtrinsic[8], originExtrinsic[9], originExtrinsic[10], originExtrinsic[11], originExtrinsic[12], originExtrinsic[13], originExtrinsic[14], originExtrinsic[15])

const intrinsic = new Matrix3()
intrinsic.set(originIntrinsic[0], originIntrinsic[1], originIntrinsic[2], originIntrinsic[3], originIntrinsic[4], originIntrinsic[5], originIntrinsic[6], originIntrinsic[7], originIntrinsic[8])

projection matrix:

const resMatrix4 = new Matrix4().setFromMatrix3(intrinsic).multiply(extrinsic)

Suppose there is a point coordinate in 3D space.

const myPoint = new Vector3(x,y,z)

Now, I have 3 questions:

  • How to derive a PerspectiveCamera based on existing conditions (extrinsic, intrinsic, resMatrix4) ?

  • How can I tell if myPoint appears in the real camera photo?

    //if i know the perspectiveCamera
const frustum = new Frustum()
frustum.setFromProjectionMatrix(perspectiveCamera.projectionMatrix) // right?
frustum.containsPoint(myPoint)

  • How to judge the position of myPoint in the real camera photo ?

    //Is the code below correct?
const position = myPoint.applyMatrix4(resMatrix4)
const s = 1 / position.z
position.set(position.x * s, position.y * s, 1)
//position.x and position.y is the coordinate of point on the real camera photo ?
2

I found an article that I feel might have my answer.

https://ksimek.github.io/2013/08/13/intrinsic/

and …


I already have an answer to the third question:
How to judge the position of myPoint in the real camera photo ?

const resMatrix4 = new Matrix4().setFromMatrix3(intrinsic).multiply(extrinsic)

const myPoint = new Vector3(1,2,3) //As an example

const position = myPoint.applyMatrix4(resMatrix4)
const isFront = position.z > 0
const s = 1 / position.z
position.set(position.x * s, position.y * s, 1)

//the real camera photo size
const photoWidth = 2048
const photoHeight = 1536

//Judging whether it is within the correct range of the photo
const inRange = isFront && position.x>=0 && position.x<=photoWidth && position.y>=0 && position.y<=photoHeight
console.log(inRange)
3

the article seems right. the problem is, 3js PerspectiveCamera does not really support independent fx and fy (edit: you can actually try to set camera.scale.x = fy / fx to work around it, I think). since PerspectiveCamera fov is vertical, you can just use fy and be done with it. in your case,

fy = 1.2058507139999999e+03 (let’s say 1206)
photoHeight = 1536

tan(0.5 * camera.fov) = 0.5 * photoHeight / fy = 0.5 * 1536 / 1206 = 0.63681592
so camera.fov = 2 * atan(0.63681592) = 1.1341 radians, or 64.9793 degrees.

(edit: if approximating it this way is not precise enough, I guess you could just overwrite camera.updateProjectionMatrix to set exact matrix you want)

4

I found a very useful article:
Simulating Real Cameras using Three.js
https://segments.ai/blog/simulating-cameras-three-js

1 Like