Managing shared resources

The game engine I am building uses a large number of graphic resources: textures, materials, buffers and so on. Many of these resources are shared, because we want to avoid loading the same texture or resource twice. So for example the noise texture used in animating the ocean waves is also used to interpolate between cobblestones and grass in the terrain shader. Having each of those shaders load the texture separately would mean using twice the GPU memory.

In order to avoid an architectural nightmare where every shader has to be aware of the lifetime and ownership needs of all the other shaders, I have implemented a caching and reference counting system. For textures, there is a TextureCache class that uses the lru-cache npm package. The entries in the cache are little wrapper objects - I call them TextureRefs - which hold a reference to the resource as well as a reference count. When the reference count fall to zero, the texture is disposed via .dispose().

Note that the cache itself counts as a reference - when the entry is added to the cache the reference count is incremented, and when the entry is evicted from the cache (using the provided lru-cache hook) it is decremented. This means that the initial reference count of a texture will be 2 - one for the cache, and one for the caller requesting the texture. It’s the caller’s responsibility to call .release() on the object when it is no longer needed.

However, there are a couple of drawbacks to this approach. First, having to use wrapper objects everywhere instead of the plain three.js objects is a bit cumbersome. Second, it means that my code isn’t very modular or portable - I can’t take my terrain generator or my particle engine and split it out as a separate npm package, because those modules are written to depend on the caching and reference-counting framework.

So my question is, has anyone faced similar issues, and do they have a better approach?

BTW, here’s an example:

This video shows a number of particle emitters creating using my “artistic particle engine”, which allows particle effects to be authored interactively in the game editor. The two emitters on the right - the colored smoke and the natural smoke - are both using the same “puff of smoke” texture. During play, the particle emitters get created and destroyed dynamically as you move around the landscape - get within 30 meters of a campfire and a smoke particle emitters gets created. This means that the number of emitters is constantly in flux, which in turn means that the number of owners of the particle texture is also in flux.

I’ve thought it would be nice to separate out the particle engine as a separate package so that other people could take advantage, but I see no easy way to do this.