I’d like to introduce one of my spartime projects i’m wroking on. It’s going for about a year now and slowly becomes complete. I’ll drop updates of progress as it proceeds.
About a year ago i’ve started working on a terrain engine mainly for volumetric terrain. A half year ago i then started another focusing on rendering realscale planets with very low memory footprint and reasonable performance for low-end machines and mobile devices. It first was more of a prototype to see how practical it can be done for realworld applications/games, and since i didn’t used papers, it also was a fun challenge to solve the issues you face in developing a planetary renderer.
Some dev clips. The game i’m actually using it for has a simple toon style, i’ll setup a realistic demo environment soon. I disable culling for tests, several post effects or quick motions made the recording tool pixelate/struggle a little too
Being fast, scalable and flexible are one of the main goals, as it also should run on low-end machines aswell as mobile devices, while requiring less as possible workload, giving the room for actual game mechanics. From resolution of datasets, topology quality, geometry detail, view distance, lod-techniques everything is adaptive to the required targets.
The LOD depth basically as no limit from a meters to centimeters resolution it depends on the usecase. A low data resolution and higher LOD depth performs and scales better. Sources can hit precision limits depending on their operating area, that’s why they are localized. A 400 KM source-probe for example can already give very high precision while global sources are mostly relevant into things like weather or rough continents. The example video above uses a global source for the continent shapes.
I’ll further describe few things below…
Procedural designed worlds
The goal of this engine is to create universe scale game worlds based on both, procedural generated datasets as well as handcrafted. The tools for designing range from procedural brushes, simulating brushes, cloning stamps to more advanced structure pattern designing, what allows the designer to control the look and characteristics of entire realscale landscapes to make them believable for the desired style or theme, within a reasonable timeframe.
Volumetric terrain
Caves, cliffs and overhangs are done via dual contouring for an efficient adaptive geometry generation. 3D features are localized, that means only where volumetric sources are, such batches are processed, a hybrid of fast GPU based 2D terrain and volumetric features. Those features require CPU processing, while the GPU terrain is instant also for fast movements or immediate jumps.
CPU & GPU Rendering
I’ve implemented a CPU Renderer which transpiles GLSL to optimized JS shaders. The main purpose is to pick single samples or render in workers or server environments. This also is used for raycasting in cases, where collision buffers aren’t available since 2D geometry is on the GPU, only close terrain will be read back for precise face collisions.
Multi-Threading for tasks
Tile and data generation is always GPU accelerated and works asynchronous by distributing the workload across render-cycles. Models and their attachments can be replicated to threads, the components implement their specific optimized synchronization, this is useful for more CPU intensive tasks.
GPU accelerated multi-layer content generation
The IndexedVolume has a layers and chunks architecture, layers are basically separate octrees for different kind of content, chunks are to dynamically load/stream and save larger portions of the worlds content. By default there is a dynamic layer for active changing objects such as players, enemies or user created content and a static layer that is purely to be used by static assets such as foliage placed automatically.
Content generation is fully GPU accelerated, from distribution, selection to placement. The assets repository is automatically indexed creating a stubs LUT of the IDs and their biome values, so they can be naturally selected by the biome data.
Multiple layers with different chunk level (sizes) and ranges/cache ranges can be defined, tags are given to make the specific stubs LUT with all assets that are available to this layer. A layer may use volume-indexing (inserting the objects into the static layer of the IndexedVolume) or use layer-indexing which is the fastest possible only culling by the tile that casted it, for the highest LOD layers such as grass and smaller foliage, insertion is instant.
Sources
These localized or global puppies actually feed the generator to write tiles of datasets. These can do blending operations with each other, use a falloff and access other datasets to generate their own, like depth or normals. Sources can be locally restricted, so each tile only renders sources containing them, the falloff mask additionally reduces areas the source doesn’t contribute to.
This also goes for pixel density, for example sources describe continents with millions of smaller details, which stream i/o based on by the LOD. This is similar to how landscapes are brushed, which are similar to how world map systems like Google Maps work.
Using satelite data
I hadn’t time yet to use the Mapbox API for some experiments, but basically it’s quite easy to use satelite data. I investigated into the Mapbox API, as they are free for a certain traffic limit and provide depth and normal tiles. I’ll look into that soon or later.
Datasets
Datasets are defined for the tile-manager and are used for any kind of spatial information, such as depth for heightmap data, biomes or simulating weather. The atlases are automatically managed, merged for multi-framebuffers, and use a chunksize according to what the GPU works good with for example.
Scene management
The scene is managed and renderered through the IndexedVolume component. This provides an interfaces for a lot usecases like efficient object access, collisions, but also will hierarchically cull nodes of objects. A impostor renderer will turn distant objects into 3D impostor, and a dataset module to render implyed massive dense areas of assets like forests by using their average color is used.
Images from development
A massive flattening probe, to flatten and level an area to build a town on.
Dynamic weather component with realtime clouds also using the atmosphere system below, the idea is to make it an all in one solution to also enable hybrid usage such as semi-volumetric from bottom transitioning into full volumetric approaching them. The clouds seamlessly soft-fade intersecting mountains or anything else in the scene.
Automatic global biomes, a global configurable probe.
A new fast physical abstracted atmosphere, the atmosphere is rendered to a cubemap which is supposed to be used as environment map.
Map maintaining it’s own state/tiles with different settings, but sharing the underlaying contents such as content and probes attachments, data-layer setup, materials, biome etc. The max. depth is cut in half reducing details, it directly uses the original pipeline of data-layers and a game specific stylized shader. It’s also going to render roads/paths and display labels and symbols of contents depending on zoom-level.
2018 / 07 / 06
Recent half year i was rather busy with work and other things, but also worked on content and collision feedback, which is a quite complex task to efficiently enable precise polygon collisions.
Besides some more optimizations i’ll implement a weather module, including simulation for wind and temperature. I’m also adding volumetric clouds similar to the game “Horzion Zero Dawn” and generally realistic materials and atmosphere by default. But i rather focus on performance than making everything phsyically correct.
One important aspect of the engine is having global information, such as using wind to influence objects, or objects influecing water with waves or grass, as well as for rather simple but effective global illumination.
2019 / 06 / 13
Recently been working on improving near details and restoring CSM (cascaded shadow maps), but i also want to fixer shimmering and transitions for it. For large scenes CSM simply are a requirement.
Using texture arrays also improved the quality for terrain materials, the default atlas technique will be only used as fallback.
I’m currently adding a default system for generating paths/roads also with crossings. Profiles with materials can be defined and assigned to segments. I figured out a backface depth buffer technique that prevents z-fighting and geometry conflicts even with flat plane profiles on the terrain.
Also added parallax occlusion as terrain material option.It doesn’t require any more data than the depth/bump map and works out of box, but is quite more costly in terms of performance. I’m also thinking about adding a modular way for adding custom shaders, this way raymarched materials could be used such as for grass, it would split the affected patches to be a separate drawcall.
para
2021 / 06 / 4
I’ve been quite busy recently and the project that uses the engine also consumes time, i might start a patreon soon so i can spend more time on it as it’s a freetime project, with a recent milestone it shouldn’t be too much longer.
I recently finished a couple more technical things such as a tech i made years ago, but couldn’t get to work at that time. It creates a tiny 8bit map of assets for “Volume Hull Impostors”, which are 3D impostors that have the same visual appearance as the original model, but don’t consume a huge amount of memory, which allows every larger asset to be rendered as such.
Here’s a example, the lantern with the axis helper is rendered as mesh closer, and as impostor once the Y axis is visible.
An here in another test (trees in distance are impostors), another feature i added to the static component is some sort of density based ambient occlusion, as even though every tree casts shadows, it wouldn’t give the feel of a real dark forest even if it’s day.
The terrain material also has a improved depth based splatting now, as with the technique to enable up to 256 materials it was a bit more complicated than i thought 
Materials with decoration sets such as grass also will derive their tint from the blending ground
The paths/roads systems also will use the bump/depth texture in order to mask with the terrain but it won’t compete with the terrains materials, rather height influences can alter parts being more or less visible having more variation then, or for open paths to blend out.
