Rendering a 3D Cuboid onto a Camera Image Using SVG in Three.js

Questions
1

Hi everyone,

I’m working on a project where I need to render a 3D cuboid onto a camera image using Three.js and SVG. Here are the details:
3D Cuboid Data:

{
  "id": "c90193ab-1812-4b52-a853-030d44a901fd",
  "objType": "3D_BOX",
  "viewIndex": 0,
  "center3D": { "x": 23.708989938369655, "y": -4.249143642992031, "z": 0 },
  "rotation3D": { "x": 0, "y": 0, "z": -1.5707965497209813 },
  "size3D": { "x": 2.0937808419650805, "y": 2.832762406310554, "z": 0.2 }
}

Camera Calibration Details:

{
  "rgb_highres_left": {
    "type": "camera",
    "image_url":"localhost:3000/image.jpg"
    "stream_properties": {
      "intrinsics_pinhole": {
        "camera_matrix": [7265.09513308939, 0.0, 2099.699693520321, 0.0, 0.0, 7265.09513308939, 1217.709330768128, 0.0, 0.0, 0.0, 1.0, 0.0],
        "distortion_coeffs": [-0.0878118, 0.0196239, 0.0, 0.0, 0.902557],
        "width_px": 4112,
        "height_px": 2504
      }
    }
  }
}

Coordinate Systems:

{
  "rgb_highres_left": {
    "type": "sensor",
    "parent": "base",
    "pose_wrt_parent": {
      "translation": [0.0609096, -0.136682, 3.51522],
      "quaternion": [-0.0121052, 0.0606543, 0.188524, 0.980119]
    }
  }
}
  • I need to render the 3D cuboid onto an image taken by the camera.
  • The rendered image should be in a width of 600 pixels. The height should maintain the aspect ratio of the original camera image.
  • The cuboid’s 3D position, rotation, and size need to be accurately represented in the 2D SVG image based on the camera calibration and coordinate system details.

Could someone guide me on how to achieve this in Three.js? Specifically, how do I map the 3D coordinates to the 2D image plane and then render the cuboid onto an image? Any code examples or pointers would be greatly appreciated!