(Streamable) level data format. Ideas?

Hi folks, it’s been a while :smiley:

I’ve been picking up my game (and engine) I’ve been working on-and-off for the past 5 or 6 years and I’m currently brainstorming about a way to store level data (in memory) that allows streaming content based on the position of the player. Think of a Minecraft world where only “cells” are drawn that are around the player.

My game uses reusable entities. Think of them as “tiles”, of which each has an ID reference which could just be a number. Ideal for storing in an array buffer. In reality, these entities are simply an Object3D instance which can contain pretty much anything. Ideally, these tiles all have the same size.

So my idea was to create a (large) array buffer that contains the entire world, and simply read data from it based on the position of the player. So, if the player can see 32x4x32x tiles (x, y, z) at any given time, that would mean running for-loops over 4096 elements every time the player moves. In my opinion this probably isn’t the best way to go about this… If the grid would be 2D then it wouldn’t be such a problem, but since height positions should also be stored, the amount of elements to iterate over every time becomes a lot bigger.

I’m wondering if anyone has any other ideas on how to approach this? Mabye an ArrayBuffer isn’t the way to go? I’m curious about some form of data format that would allow “grabbing” a set of elements based on spatial position.

The end-goal is to create a possibly infinite world that (is / could be) streamed in by the server.