Calculate Inverse Vector according to camera position

Questions
1

I have two scenes. Scene A - which contains a camera and equirectangular image projected onto a large sphere. And Scene B - which is the same.

A user can click to go from Scene A and Scene B. I store this position of the click, and I translate the Sphere of Scene A TOWARDS the camera according to the vector of the click, and when the user arrives at Scene B, the Sphere translates FROM behind the camera. The result is a “walk through” effect between scene A and Scene B. Heres a video: Loom | Free Screen & Video Recording Software

This works great ONLY if the two scenes have cameras mostly in the same position. Sometimes the direction of the camera in Scene A is looking at a different direction than the camera on Scene B.* In such a scenario, this effect fails. Heres a video: Loom | Free Screen & Video Recording Software

Spot the difference? In the second video, the Scene B translates according to a random starting position, whereas in the first video, Scene B always appears to translate along the same direction of the Scene A translation.

I have tried a bit to modify my code (spent probably 20 hours on this). I have tried a lot of things, but I think I am missing some fundamental understanding.

Can anyone point me in the right direction? Thank you.

Here is the code I am working with:

       // get last raycast and last camera position
    let lastTouch = sceneA.lastTouch;
    let lastCameraPosition = sceneA.camera.position;

    // get a direction vector to travel upon that is the distance between raycast and last camera position
    var dir = new THREE.Vector3().subVectors(lastCameraPosition, lastTouch).normalize();

    // Move Scene A Sphere 2000 units along dir vector
    let sceneA_END = dir;
    var leaveSceneAAnimation = new TWEEN.Tween(sceneA.panorama.position)
        .to({ x: sceneA_END.x * 2000, y: sceneA_END.y * 2000, z: sceneA_END.z * 2000 }, 1500)
        .easing(TWEEN.Easing.Cubic.Out);// Linear.None

    // Inverse Vector to prepare for Scene B entrance
    let sceneB_START = dir.clone().multiply(new THREE.Vector3(-1, 0, -1));
    let sceneB_END = sceneB.panorama.position.clone();

    // set Scene B Sphere 2000 units away from where it should end up
    sceneB.panorama.position.set(sceneB_START.x,sceneB_START.y,sceneB_START.z);
    sceneB.panorama.position.needsUpdate = true;

    // Move Scene B Sphere 2000 units along inverted dir vector
    var enterSceneBAnimation = new TWEEN.Tween(sceneB.panorama.position)
        .to({ x: sceneB_END.x * 2000, y: sceneB_END.y * 2000, z: sceneB_END.z * 2000 }, 1500)
        .easing(TWEEN.Easing.Cubic.Out);// Linear.None

    leaveSceneAAnimation.start(); 
    enterSceneBAnimation.start();

*Note: The reason the cameras are sometimes in different directions is that the camera must rotate to accomodate a real world north point of an image - when scene images are captured, scene A might be captured facing NorthEast, and Scene B might be captured facing SouthWest. We rotate the Three.js scene camera to account for this.

Also, if someone has a better title idea for my question, I would really appreciate it!

2

/cc

3

One idea I had is to get the camera rotation of Scene A and scene B, and to calculate the offset rotation:

let offset = fromScene.control.getRotations().h - toScene.control.getRotations().h

This I tried applying the offset to the original travel vector from Scene A, and use this for Scene B:

let sceneB_START = dir.clone().applyAxisAngle(new THREE.Vector3(0,1,0), offset)

But this didn’t work. Seems like it should? Perhaps I do not fully understand vectors - though I am trying.