Postprocessing mesh interaction

Questions
, , ,
1

Hi guys, right upfront: I’m relatively new to threejs and webgl.

I implemented a camera pincussion distortion effect using post-processing which is working fine. The problem now is that the listeners of the meshes rendered within my scene seem to not adapt to the distorted view.

Example: I have a few tiles rendered in the xy plane facing the camera. After applying the pincussion effect the ones that are futher away from the camera lookAt vector seem to be closer (edges get bend towards the camera. The issue is now that I have to check for user interaction with those tiles. Clicking on a tile that is not directly faced by the camera will not fire the event listener. However clicking on the positions where the tiles have been before applying the distortion fires the event listeners.

Is there any way to update the meshes or the scene accordingly to the distorted post-processing scene grid so that I can actually click on the tiles?

If this is not the case is there a way to distort the entire coordinate system of the scene to appear “curved” in xy to create a similar effect?

Thx in advance. Any help is appreciated.

2

Hm. What about this idea that came to me out of the blue:

  • left is the original scene, undistorted, this is the scene which is used by the raycaster
  • right is the distorted scene, which is shown on the screen and mouse clicks are within this distorted coordinates

The idea:

  • get the screen coordinates of the mouse click (right image)
  • convert them back into original coordinates (i.e. barrel transformation that will undo the pincushion transformation)
  • now you have coordinates at the left
  • use raycaster with these barrelled mouse coordinates

Here is my poor attempt to visualize the idea:

Untitled Diagram

And also

Off-topic info

3

fiber allows you to set the event compute, this is also how portals and so on manage to get events, even on object UVs like this Using decals - CodeSandbox.

Function PincussionEvents() {
  const get = useThree(state => state.get)
  const setEvents = useThree(state => state.setEvents)
  useEffect(() => {
    // Fetch current compute
    const oldCompute = get().events.compute
    // Set new one
    setEvents({ compute: (event, state) => {
      // Every event system-wide will pass through here ...
      // The computes job is to set the pointer and the raycaster
      state.pointer.set((event.offsetX / state.size.width) * 2 - 1, -(event.offsetY / state.size.height) * 2 + 1)
      state.raycaster.setFromCamera(state.pointer, state.camera)
    }})
    // Put old event compute back on unmount
    return () => setEvents({ compute: oldCompute })
  }, [])
}

<Canvas>
  <Effects />
  <PincussionEvents />

i wouldn’t know how, but you need to bend the event pointer as well, or unbend. use what @PavelBoytchev has outlined above and wire it into the new compute. and et voila event will start working again.

2 Likes