Advice on optimizing portals

[viridia]

As some of you may recall, I implemented a portal system in three.js a couple of years ago, as I described in this medium post: Creating portals in three.js. In this article, I’ll show you… | by Talin | Machine Words | Medium

One of the things I have noticed recently is that portals appear to be the biggest single performance bottleneck in my engine. On my Macbook M1, I normally get a framerate of about 144 fps viewing a complex scene, but even a single, relatively small portal on-screen drops that to around 70fps, and 2 or 3 portals drops it even further. This is not surprising given that each portal requires a traversal of the scene graph.

Also, the way I implemented portals - using the stencil buffer - has some drawbacks, it means that translucent objects in front of the portals don’t look right. Solving this problem would require splitting the main render pass into two separate phases, so that the portal rendering pass happens after opaque objects get rendered but before translucent ones do. This is likely to impact performance even further.

I’ve been considering re-doing portals to render to off-screen textures rather than using stencils. The portal content could then be drawn before the main render pass starts, and the portals would be like any other textured object so translucent shapes in front would not be an issue.

However, my concern is the extra cost of allocating and deallocating offscreen buffers. Because the size and shape of the portal aperture changes based on the viewing angle, I’d want the allocation to be at least somewhat dynamic. I suspect that re-allocating the portal buffer every frame is probably not a good idea, but neither is allocating a giant buffer that is the same size as the main window, given that there are often as many as 4 portals visible at a time, that seems like a lot of memory.

I’d be interested if anyone had some advice as to how to make this as fast as possible, before I start writing a bunch of code based on my incomplete knowledge.

[anidivr]

Render targets might be a better solution now. They seem to perform very well.

Rather than changing based on angle, could you use a wide field of view that works well from most angles?

[viridia]

Well, I went ahead and implemented render targets, and it kind of works. The problem is that occasionally when there are multiple portals on-screen, I see garbage pixels being rendered in some of the portals:

As you can see in the screenshot, some of the pixels are correct. I thought perhaps maybe it was leftover stuff in the depth buffer, but I checked that I am clearing all buffers (color, depth, stencil) correctly before rendering to the texture.

      renderer.setRenderTarget(this.renderTarget);
      renderer.setViewport(this.portalViewport);
      renderer.clippingPlanes[0].copy(this.clippingPlane);
      renderer.clear(true, true, true);
      renderer.render(this.destinationScene, this.portalCamera);
      renderer.setRenderTarget(null);

[Antonio]

This is great! I mean, having the chance to talk about porting / implementing a different approach to this represents a massive value for the group, thanks @viridia for opening this discussion

Please consider changing the post label to ‘discussion’ (the black colored one)

I am aware that mantaining and/or keeping up to date two versions would be too much work and time, hopefully the older version could be archived instead of replaced by the newer one.

So far I have not used the initial implementation, but got some experience with render targets.
Only thing I’m gonna say by now is that you can use RenderTargetInspector from @Fyrestar to debug them, it saved me a lot of time last time I used it

[anidivr]

Just making sure you are aware of three.js examples portal using render targets

[viridia]

I found a workaround for my earlier problem (don’t render other portals from within a portal - in other words, the system doesn’t support recursive portals). However, I’ve run into a different interesting problem with lighting.

In my game engine, a portal can point to a location on the same “level” (world map) or a different level. Each level is a Scene, so the portal will render either the same scene or a different scene.

Each scene has a single sunlight light source. What I have noticed is that the shadows when viewed through the portal are sometimes incorrect - sometimes objects on the near side of the portal will cast shadows that are visible in the far side of the portal. (This might sound like a good thing, but it’s not because the shadows are not aligned in the way you would want).

What I would expect is that the shadows on the near and far sides of the portal would be completely unrelated, since they are separate scenes.

My current theory is that the shadow buffer for the sunlight isn’t being cleared during the portal render phase, and is somehow getting stale data from the previous render. However, I’m not sure how to test this theory experimentally.

Given that shadow buffers are expensive (both in terms of memory and CPU), I want to make sure that any solution doesn’t cause excessive recalculation of shadows.