readRenderTargetPixels on R3F/drei useFBO hook

Questions
1

I have the following R3F component that receives a ref to a plane mesh and its dimensions as props. The idea is that it aligns an Orthographic camera up to the plane and renders it to an FBO using R3f/drei’s useFBO hook. In the code, I’m outputting the FBO.texture to a plane mesh for debugging purposes and this displays correctly.

What I actually want to do is utilise gl.readRenderTargetPixels() to get the pixel values out of the FBO and write them to pixelBuffer. However, when I log the pixelBuffer it is filled with 0’s. I’ve searched for answers and found a few others with this problem which seemed to be caused by the order of operations but I’ve tried everything I can think of and the result is always the same.

Any help would be greatly appreciated!

function MyComponent({
	objectRef,
	dimensions,
}: {
	objectRef: RefObject<Mesh>;
	dimensions: { width: number; height: number };
}) {
	const { gl, scene } = useThree();

	const fboWidth = dimensions.width * 1000;
	const fboHeight = dimensions.height * 1000;

	const fbo = useFBO(fboWidth, fboHeight, {
		format: RGBAFormat,
		type: UnsignedByteType,
	});

	const orthoCamera = useRef<OrthographicCamera>(null!);
	const pixelBuffer = new Uint8Array(fboWidth * fboHeight * 4);

	useEffect(() => {
		const obj = objectRef.current;
		const camera = orthoCamera.current;

		if (!obj || !orthoCamera) return;

		camera.left = -dimensions.width / 2;
		camera.right = dimensions.width / 2;
		camera.top = dimensions.height / 2;
		camera.bottom = -dimensions.height / 2;
		camera.updateProjectionMatrix();

		const normal = new Vector3(0, 0, 1).applyEuler(obj.rotation);
		camera.position.copy(obj.position);
		camera.position.add(normal);
		camera.lookAt(obj.position);
	}, [objectRef]);

	useEffect(() => {
		gl.readRenderTargetPixels(fbo, 0, 0, fboWidth, fboHeight, pixelBuffer);
		console.log(pixelBuffer);
	}, [fbo]);

	useFrame(() => {
		gl.setRenderTarget(fbo);
		gl.render(scene, orthoCamera.current);
		gl.setRenderTarget(null);
	});

	return (
		<>
			<mesh position={[0, 25, 0]}>
				<planeGeometry args={[dimensions.width * 10, dimensions.height * 10]} />
				<meshBasicMaterial map={fbo.texture} side={DoubleSide} />
			</mesh>
			<orthographicCamera ref={orthoCamera} position={[0, 0, 0]} near={0} far={100} />
		</>
	);
}
2

https://disboard.org/server/join/740090768164651008

You want to do the readRenderTargetPixels right after your render.