Get orthographic 3d point from mouse position and depth map

Hello, I managed to get the position with a perspective camera based on this information: https://stackoverflow.com/questions/44121266/compute-3d-point-from-mouse-position-and-depth-map

But I am trying to alter it to work with Orthographic cameras and I am a bit stuck.
I found the Z distance from the camera, but I’m not sure how to use this to get the world space position.

function renderDepthTarget(){
				renderer.setClearColor(0xffffff, 1);
				renderer.clear();
				scene.overrideMaterial = depthMaterial;
				renderer.setRenderTarget(depthTarget);
				renderer.render(scene, camera);
				scene.overrideMaterial = null;
				renderer.setRenderTarget(null);
		
				renderer.readRenderTargetPixels(depthTarget, mouse.screen.x, window.innerHeight-mouse.screen.y, 1, 1, rgbaBuffer);
				z = v4.fromArray(rgbaBuffer).multiplyScalar(1 / 255).dot(unpackFactors);
				if(camera.isPerspectiveCamera){
					z = ( camera.near * camera.far ) / ( ( camera.far - camera.near ) * z - camera.far );
					//z now contains distance from camera
					projInv.getInverse(camera.projectionMatrix);
					depthPosition.set(mouse.screen.x / window.innerWidth * 2 - 1, -(mouse.screen.y / window.innerHeight) * 2 + 1, 0.5).applyMatrix4(projInv);
					depthPosition.multiplyScalar(z / depthPosition.z);
					depthPosition.applyMatrix4(camera.matrixWorld);
					//depthPosition now contains the world position
				}else if(camera.isOrthographicCamera){
					z = z * ( camera.near - camera.far ) - camera.near;
					//z now contains distance from camera
					
					//not sure what to do here
				}
			}

Do I need to do something like the following?:

depthPosition.set( mouse.x, mouse.y, ( camera.near + camera.far ) / ( camera.near - camera.far ) );
depthPosition.unproject(camera);

Although I don’t know how the Z works into that, if that is the case…

Any guidance is appreciated.