Convert camera frustrum to UV coordinate on texture?

Hello! First off, I’m not sure I’m asking this the right way. Please bear with me as I explain what I’m trying to do.

I’m looking to implement something similar to DZI (deep zoom image) but for panoramas. My current solution is creating a large number of sphere slices (adjusting phi and theta values) and texturing them with image tiles. This works for the most part but I think it uses more memory than necessary since I’m creating a bunch of sphere geos?

To optimize, I’m changing it to using only one sphere geometry with a large initial white dummy texture. And then using copyTextureToTexture() function to load tiles.

Finally, my question… I’m stuck trying to figure how to to detect which part of the sphere geo the camera is looking at to so I can update the corresponding tile. Any help is appreciate it!

You could use Camera.getWorldDirection() to get the view direction and then use raycaster to find out the intersection point on your sphere. To be clear, the origin of the raycaster’s ray would be the camera’s position in world space, the direction of course the mentioned camera’s view direction.

An intersection points always contains the uv coordinates of the intersected geometry. You can access it like so:

const intersects = raycaster.intersectObject( object );

if ( intersects.length > 0 ) {

    const intersection = intersects[ 0 ];
    console.log( intersection.uv );

}

Did not know intersections also contains uv coordinates. That would make a lot sense! Thanks!

Hello, I am pretty new to Three JS and that is exactly what I am trying to do, however so far when I try to sharpen locally my equirectangular panoramic by using copyTextureToTexture() function to load a tile nothing seems to happen.

  const container = document.getElementById("container");

  camera = new THREE.PerspectiveCamera(
    75,
    window.innerWidth / window.innerHeight,
    1,
    1100
  );

  scene = new THREE.Scene();

  const geometry = new THREE.SphereGeometry(500, 60, 40);
  // invert the geometry on the x-axis so that all of the faces point inward
  geometry.scale(-1, 1, 1);

  const texture = new THREE.TextureLoader().load(
    // This is my low res full panoramic
    "textures/pano/00001-pano_256.jpg"
  );
  const material = new THREE.MeshBasicMaterial({ map: texture });

  const mesh = new THREE.Mesh(geometry, material);

  scene.add(mesh);

  renderer = new THREE.WebGLRenderer();
  renderer.setPixelRatio(window.devicePixelRatio);
  renderer.setSize(window.innerWidth, window.innerHeight);
  container.appendChild(renderer.domElement);


  document.addEventListener(
    "dblclick",
    function () {
      const tile = new THREE.TextureLoader().load(
        // This is my high res first tile
        "textures/pano/00001-pano_4096-tex-r3-00.jpg",
        function (tiletexture) {
          texture.repeat.x = 4;
          texture.repeat.y = 2;
          texture.wrapS = texture.wrapT = THREE.RepeatWrapping;
          renderer.copyTextureToTexture(
            new THREE.Vector2(),
            tiletexture,
            texture
          );
        }
      );
    },
    undefined,
    function (err) {
      console.error("An error happened.");
      console.log(err);
    }
  );

Is there some function I am supposed to run after using the copyTextureToTexture() in order to refresh my initial texture ?
The texture.repeat and the texture.wrapS/wrapT update applies but I can’t see my first hi res tile mapped on the sphere.

you would not see it even if you did everything right since at the end of the day texture is going to be still 256 pixels, no?

Indeed you’re right my initial texture is still 256 pixels.

  document.addEventListener(
    "dblclick",
    function () {
      const tile = new THREE.TextureLoader().load(
        // This is my high res first tile
        "textures/pano/00001-pano_4096-tex-r3-00.jpg",
        function (tiletexture) {
          initialTexture.repeat.x = 4;
          initialTexture.repeat.y = 2;
          initialTexture.wrapS = initialTexture.wrapT = THREE.RepeatWrapping;
          renderer.copyTextureToTexture(
            new THREE.Vector2(),
            tiletexture,
            initialTexture
          );
          console.log(tiletexture.source.data.currentSrc);
          //   http://localhost:5173/textures/pano/00001-pano_4096-tex-r3-00.jpg
          console.log(
            `${tiletexture.source.data.height}*${tiletexture.source.data.width}`
          );
          //   1024*1024
          console.log(initialTexture.source.data.currentSrc);
          //   http://localhost:5173/textures/pano/00001-pano_256.jpg
          console.log(
            `${initialTexture.source.data.height}*${initialTexture.source.data.width}`
          );
          //   128*256
        }
      );
    },
    undefined,
    function (err) {
      console.error("An error happened.");
      console.log(err);
    }
  );

Not sure however how I am supposed to resize and reshape my texture so that it fits my tiles. I thought tuning the repeat prop and wrapS/wrapT would handle that actually.

By the way here’s my initial pano and one of my tile

Low res texture : 00001-pano_256
00001-pano_256

Hi res texture tile 00001-pano_4096-tex-r3-00

it is more like you are supposed to create a tile geometry (a curved “plane”) and add it to your sphere with the tile texture

Thank you very much for your answer. I’ve been able to create a partial sphere on which I have mapped a tile of my equirectangular panoramic


    const phiStart = 0;
    const phiEnd = Math.PI / 2;
    const thetaStart = 0;
    const thetaEnd = Math.PI / 2;

    const tileGeometry = new THREE.SphereGeometry(
      500,
      600,
      40,
      phiStart,
      phiEnd,
      thetaStart,
      thetaEnd
    );

    // invert the geometry on the x-axis so that all of the faces point inward
    tileGeometry.scale(-1, 1, 1);

    let tileTexture = new THREE.TextureLoader().load(
      "textures/pano/00001-pano_4096-tex-r3-00.jpg"
    );

    const tileMaterial = new THREE.MeshBasicMaterial({ map: tileTexture });
    tileMaterial.depthTest = false;

    const tileMesh = new THREE.Mesh(tileGeometry, tileMaterial);
    tileMesh.renderOrder = 1;

    scene.add(tileMesh);

That being said, the initial idea was to manage and handle a single mesh and to update that single mesh by copying tiles on its texture whenever I need to sharpen the resolution or to change the displayed panoramic. With that approach I would have to handle potentially 33 spheres (32 partial spheres tiles + the very low res full sphere) because I plan to split my panoramic into 32 tiles. Wouldn’t that be a potential performance issue ?