Compile multiple scenes with different cameras for performance reasons

Questions
, , ,
1

Hi all,
in my R3F project, I have multiple scenes, each with their own camera rendering to a renderTarget (set up in SceneComponent.jsx). In my main scene, I show these scenes and the transitions between with a shader material based on the scroll position. The main scene gets rendered with the default camera.

How would I go about compiling my custom scenes with their own cameras, other than the default scene from the Canvas component? I want to eliminate the stutter at the first transition which you can see in the video.

The important part of my MainScene.jsx look like this:

useFrame(({ gl, scene, camera }) => {
    gl.setRenderTarget(null);
    gl.render(scene, camera);
    materialRef.current.progression = props.globalProgress.current;
  });

return (
  <>
    <mesh scale={scale}>
      <planeGeometry />
      <transitionMaterial
        ref={materialRef}
        progression={props.globalProgress.current}
        tex0={globalScenes[0].renderTarget.texture}
        tex1={globalScenes[1].renderTarget.texture}
        tex2={globalScenes[2].renderTarget.texture}
      />
    </mesh>

    {globalScenes.map((scene, index) => (
      <SceneComponent
        key={index}
        renderTarget={scene.renderTarget}
      ></SceneComponent>
    ))}
  </>
);

The custom SceneComponent.jsx roughly looks like this:

    const { cameras, animations } = useGLTF(props.model);
    const scene = useRef();
    const { gl } = useThree();

    useEffect(() => {
      useGLTF.preload(props.model);
      // gl.compile(scene, cameras[0]);
      // This leads to TypeError: targetScene.traverseVisible is not a function
    }, [props.model]);
  
    useFrame(({ gl }) => {
      // if (!props.active) return;
      gl.autoClear = true;
      gl.setRenderTarget(props.renderTarget);
      gl.render(scene.current, cameras[0]);
    });
    
    return (
      <>
        <scene ref={scene}>
          <group ref={group} dispose={null}>
            {props.sceneJSX}
          </group>
        </scene>
      </>
    )

I can’t wrap my head around how to compile these scenes, if I try to do gl.compile() and pass in the sceneRef and the camera which I got from the GLTF file, I get a TypeError: targetScene.traverseVisible is not a function

I’m also unsure if the way I set this up is valid at all. For instance, I noticed that in the first scene the EnvironmentMap reflections don’t show up until the stutter, when the next scenes get compiled (also visible in the video).

Any hint as to how to use gl.compile() in R3F would be much appreciated!

2

this is all taken care of, you can delete all setRenderTarget stuff. check out the portals section of drei GitHub - pmndrs/drei: 🥉 useful helpers for react-three-fiber

you have

  • Hud
  • View
  • RenderTexture
  • RenderCubeTexture
  • Fisheye
  • Mask
  • MeshPortalMaterial

all these tap into fiibers createPortal, which creates a sealed, isolated canvas-like environment inside a rendertexture, this includes events, you can drop your environments in, controls etc.

to get an impression try these:

but this should cover it:

function Foo({ scene1, scene2, scene3, ...props }) {
  return (
    <mesh {...props}>
      <planeGeometry />
      <transitionMaterial>
        <RenderTexture attach="tex0">{scene1}</RenderTexture>
        <RenderTexture attach="tex1">{scene2}</RenderTexture>
        <RenderTexture attach="tex2">{scene3}</RenderTexture>
      </transitionMaterial>
    </mesh>
  )
}

...
<Foo
  scene1={<Scene1 />}
  scene2={<Scene2 />}
  scene3={<Scene3 />} />

...
function Scene1() {
  return (
    <group>
      <PerspectiveCamera makeDefault position={[0, 0, 10]} fov={75} />
      ...
2 Likes
3

PS

you can also use raw createPortal directly btw, it’s low level but insanely powerful. the following creates a plane into which you can render a scene. the scene is alive, events, everything would work.

import { createPortal } from '@react-three/fiber'

function Foo({ children, ...props }) {
  const [scene] = useState(() => new THREE.Scene())
  const buffer = useFBO()
  return (
    <group {...props}>
      {createPortal((
        <>
          {children}
          <Render buffer={buffer} />
        </>
      ), scene)}
      <mesh>
        <planeGeometry />
        <meshBasicMaterial map={buffer.texture} />
      </mesh>
    </group>
  )
}

function Render({ buffer }) {
  useFrame((state) => {
    // everything accessed through state is *inside* the portal, including the cameras
    state.gl.setRenderTarget(buffer)
    state.gl.render(state.scene, state.camera)
    state.gl.setRenderTarget(null)
  })
}

<Foo>
  <PerspectiveCamera makeDefault position={[10, 10, 10]} />
  <mesh>
    <boxGeometry />
  </mesh>
</Foo>

with drei it would be just this, same functionality as above.

function Foo({ children, ...props }) {
  return (
    <mesh {...props}>
      <planeGeometry />
      <meshBasicMaterial>
       <RenderTexture attach="map">
         {children}
       </RenderTexture>
     </meshBasicMaterial>
    </mesh>
  )
}
1 Like
4

Hey! Thanks so much for your answer, if only I knew beforehand!
I’ll definitely give this a go this week!

Being new to both React and R3F, what I can’t figure out right now, is how to pass the render textures to my custom transitionMaterial. I see the attach=“tex0” part, but how do I prepare my transitionMaterial to “play the R3F way” and receive those attach props?

5

nothing needs to prepare for or anticipate react, fiber is threejs. if you have a normal shadermaterial and you extend it so that it works lowercase in jsx then all properties will be accessible.

quickly read through this, it will give you some peace of mind React Three Fiber Documentation down below it talks about attach as well

1 Like
6

I just got it working :smiley: No more stutter, lighting works fine and the code overall is way cleaner. Thank you so much, you made me very happy!

1 Like
7

Hi, could I ask how you made the transitions (transition effects) between the different scenes?
Maybe you have a take on this as well @drcmda?