i’ve tried porting the antimatter impl which was also used by aframe, this is how a renderable would behave:
so everything is now under the users control, the type of camera, the scene, controls. but it’s lacking unified instancing for more than 1 splat. it would be silly easy from reacts perspective to manage that, but the shader stuff and the worker is quite overwhelming, it looks like the antimatter codebase hasn’t prepared for that yet.
Thanks for the example! I think making my code work in a similar fashion will be pretty easy, but supporting multiple splat scenes is going to be the tricky part. Is it worth releasing a drop-in version or R3f version that only supports a single splat scene?
i think i will attempt managing multiple splats today, i guess it’s just about pumping bufferarrays into the worker. though removing splats is probably going to be hard. i would suggest you keep it vanilla as extensible functions. instead of an engine people will have to create a splatmanager class and interface the renderable object3d’s with it by constructor injection or something like that?
i can componetize it into fiber for you if you want a broader audience.
btw, i have never really tried this before because i thought this needs specialized hardware. but i studied a bit and turns out it’s actually super easy to create. this really is a game changer, i don’t have to be a blender artist to make showcases on the web.
the original code uses this code to position the splats
let center = new THREE.Vector3(f_buffer[8 * i + 0], f_buffer[8 * i + 1], -f_buffer[8 * i + 2])
let scale = new THREE.Vector3(f_buffer[8 * i + 3 + 0], f_buffer[8 * i + 3 + 1], f_buffer[8 * i + 3 + 2])
let mtx = new THREE.Matrix4()
let mtx_t = mtx.clone()
each splat is a virtual placeholder (a THREE.Group with nothing in it), the world matrix of it is obj.current.matrixWorld, do you know how i can apply it? i tried multiply, premultiply, add, but the results are rather crazy.
Well my code is separated into multiple files and multiple classes, so my hope is that it is easier to read and understand… however I could be totally wrong about that But I have also taken additional steps for performance optimization. I use an octree to cull splats that are not visible or near the frustum in order to speed up both rendering and sorting. I implemented my sorting algorithm in C++ and it’s imported via a WASM module. The sort itself uses web assembly SIMD instructions and is currently 100% integer math based (which may become an issue with larger scenes and the potential of overflow). I’m also using a transform feedback to calculate splat distances from the camera prior to sorting. I have also tried to document the shader code so that it is easier to understand how the covariances for each splat are projected and ultimately rasterized. Additionally I created a custom .splat file format with some basic compression that significantly reduces file size, although work on that is ongoing: Universal format discussion
I have to give Kevin Kwok credit though; he implemented his version first and it was a great starting point for mine. I have tried to get mine to where it is 100% my own code… I think I have been mostly successful
My “drop-in” branch is now nearly complete, it supports adding a renderable splat viewer to a three.js scene just like any other three.js renderable. It also supports loading multiple splat scenes and applying custom transformations to those scenes. Example usage can be seen here: Drop-in example. Let me know if this is heading in the right direction