R3F, using screen fill and replace by videoTexture

Questions
,
1

Hi there,

I’m working on a project that shows a room in 360 degrees that you can orbit, the room has a 360 degrees videoTexture.

However, before the video is loading, I want to use a screen fill that is displayed until the video is done loading.

It works, but there is a side effect which I can’t wrap my head around.

When the videoTexture replaces the screen fill it’s been Over brightened and noisy, it looks like it’s overlaying the videoTexture with the previous screen fill.

Has anyone any idea what goes wrong, do I need to dispose or clear the texture and if so, how I do this?

Hope someone can help me out thanks,

here’s the code:

  const [videoTexture] = useState( () => { 
    video.crossOrigin = 'anonymous'
    video.playsInline = true
    video.volume = 0.0
    video.loop = true 
    video.poster = globalConfig[fillname]

    return new THREE.VideoTexture(video)
  })

Happylives

2

i don’t think that’s your issue but useMemo is for memoized calculations, not side effects. by rec-creating the video on every flag change and adding listeners you already have race conditions. side effects always go into useEffect, and if you have a listener, make sure to clean up, too.

useEffect(() => {
  const fn () => ...
  video.addEventListener('foo', fn)
  return () => video.removeEventListener('foo', fn)
}, [])
3

Hi drcmda,

Thank you for your feedback, it doesn’t solve the issue, but your right about the feedback.

Happylives

4

I have edit the code based on your feedback, I also added the poster, attribute that displays the first frame while loading.

however this only works with videos so if want to display a texture image first it still gives the effect of over exposure.

Screenshot 2022-09-07 at 14.29.23
Screenshot 2022-09-07 at 14.29.23466×812 242 KB

so this is the effect that im getting.

5

i have no idea what i’m looking at, i just see a plant. if you want make a very reduced 50 lines max codesandbox to show the problem i can perhaps help.

ps “overexposure” sounds like tonemapping and encoding to me. the video texture should have encoding={THREE.sRGBEncoding} and the material toneMapped={false}. this is how you use video textures: Video textures - CodeSandbox

6

btw a fallback for video would be a good usecase for a hook in drei that integrates video load into suspense so that you can have easy fallbacks:

<mesh>
  <planeGeometry />
  <Suspense fallback={<meshBasicMaterial map={fallbackTexture} />}>
    <VideoMaterial url="foo.mp4" />
  </Suspense>
</mesh>

function VideoMaterial({ url }) {
  const video = useVideo(url)
  return (
    <meshBasicMaterial toneMapped={false}>
      <videoTexture attach="map" args={[video]} encoding={THREE.sRGBEncoding} />

how can i know that a video has finished loading? i could add this to drei quick

7

Hi,

Using the fallback is indeed a good idea, which im trying to achieve, but the fall back doesn’t
show anything just a dark Gray background.

what I want is this:

  const placeholderTexture = useLoader(TextureLoader, 'texture url') //jpg image

<Suspense fallback={<meshBasicMaterial map={placeholderTexture} side={BackSide}/>}>
 <meshStandardMaterial map={(isLoading) ? placeholderTexture : videoTexture} side={BackSide}/>
</Suspense>

It actually works, is show the placeholderTexture and then switches to the videoTexture, but over exposed and grainy:

Screenshot 2022-09-08 at 14.14.25
Screenshot 2022-09-08 at 14.14.251212×845 54.6 KB

PS. I really appreciate your effort, thanks!

8

that’s not how suspense works im afraid. can you give me a description which events i need to listen to, i dont know how to use the doms video tag. i can build that hook. as for graininess - that seems unrelated to me. what has react, hooks, suspense, fiber, threejs to do with grain in a video texture.

9 
  useEffect(() => {
    video.addEventListener('loadstart', () => {
      setIsLoading(true)
    })

    video.addEventListener('canplaythrough', () => {
      setIsLoading(false)
    })
    
    return () => {
      video.removeEventListener('loadstart', () => { })
      video.removeEventListener('canplaythrough', () => {})
    }
  }, [video, setIsLoading])

Because that is the side effect, the video it self is not grainy its being effected flipping the Placeholder with the videoTexture

that is why I found it so strange…

10

could you make that box meanwhile? i need to see it, otherwise i can’t help.

11

Hi yes here it is:

12

here’s the drei hook: useVideoTexture: Video textures - CodeSandbox

it’s integrated into suspense and now can have fallbacks. docs: GitHub - pmndrs/drei: 🥉 useful helpers for react-three-fiber

13

Yeah this works.

Thank you very much!!!
:ok_hand: