Hey
I was wondering if there was a technique available to extract the frame/fragment data from a video texture and use it at will within a scene? Such as taking the brightness values and using that to drive object positioning or something like that?
Might be missing something obvious but I couldn’t spot anything in the docs!
Cheers,
Ono
Edit: Would I be correct in assuming that the answer would be roughly the same as this one - [SOLVED] Extract pixels data from DataTexture
Yes but you don’t need to use gl.readPixels directly… there is now a renderer.readRenderTargetPixels method in threejs that lets you read pixels back out of a rendertarget.
https://threejs.org/docs/#api/en/renderers/WebGLRenderer.readRenderTargetPixels
so you would create a THREE.VideoTexture( yourVideo)…create a rendertarget of the size you want…
renderer.setRenderTarget( yourRenderTarget )
advance the video to the frame you want, render a plane with a basicMaterial( { map: yourVideoTexture } )
then renderer.readRenderTargetPixels() to get the image data out into an array.
Lovely, great answer will give that a go!
Your reply gave me a follow up question regarding size? Did you mention that purely for completeness or is recommended to set a smaller size for performance or something?
Both for completeness, and also you’ll probably want the rendertarget dimensions to match the video width/height if you want to get all the image data for a frame…
If you’re planning on doing this in realtime, then yes, you may want to use a smaller target (but with the same aspect ratio) to speed up the pixel read back. (pixel readback is a rather slow operation, so minimizing the size of data will give you some control over how much impact it has.)
(nice name btw 
)
Ah excellent, would you propose fewer reads? such as reading every other render frame or something like that? Haven’t tried it yet so I am keen to get any obvious optimisations out the way early!
Much appreciated! Mutual fan of CASE I take it
Yup I would experiment first tho. The impact will depend on … the speed of your machine, the quality of your GPU, the browser…
If this is an art piece where you have control over the deployed hardware, you can control those factors… if its for mass market, you’ll have to tune it for the lowest common denominator… (safari / ios ) 
! Thanks again