Architecture advice needed for an immersive browser-based historical art gallery

Questions
1

Hi friends,

I’m working on an immersive, browser-based art experience set inside a fictional Swiss castle around the year 1500.

I’m not a professional developer — I’m a designer and creative project owner — but I absolutely love this world! I have been learning and prototyping web experiences through practical work with ChatGPT/Codex and Claude as AI-assisted coding partners.

Over the past four weeks, I’ve put an enormous amount of time into this project — including many sleepless nights — and have used a considerable number of credits across different AI and software tools. I still love the project, but I want to stop losing time and credits on the wrong technical path. I would be genuinely grateful for honest, experienced guidance.

The creative direction, visitor experience and visual quality are my responsibility. I can follow technical explanations, test prototypes and make product decisions, but I would appreciate experienced guidance on the underlying 3D architecture and production workflow.

I’m not asking the community to write the project for me. Before continuing production, I want to understand which technical foundation is realistic and what a sensible first milestone should be.

The intended experience is more than a collection of images or panoramas. Visitors should be able to move through exterior and interior spaces, enter galleries, approach artworks, open a large artwork view with information, and submit an enquiry or reservation request.

The project should eventually include:

stable and historically controlled castle and room geometry

intuitive mouse, touch, touchpad and keyboard navigation

desktop, tablet and smartphone support

galleries with regularly changing exhibitions

precise artwork dimensions and wall positions

artwork interaction, large view and metadata

lighting, music and environmental sound

seasonal and day/evening variations

optional animated characters and atmospheric effects

A local TypeScript/Three.js greybox already exists. It contains several connected visitor stations, navigation, responsive behavior and automated tests. It was developed through AI-assisted coding with Codex and Claude under my creative and product direction.

However, the visual production approach is currently uncertain. Experiments with AI-generated viewpoints, panoramas and a generated 3D world did not preserve the castle identity or provide sufficiently stable, editable geometry.

I am now considering:

Blender-built geometry exported as GLB and rendered with Three.js

Blender-built geometry combined with PlayCanvas

Gaussian splats or panoramas only for selected atmospheric backgrounds

A hybrid approach

My main questions are:

Should the castle and gallery rooms be based primarily on real mesh geometry?

Would you continue with the existing Three.js foundation, or test PlayCanvas because of its visual editor and built-in components?

What is the best way to keep room geometry stable while changing artwork exhibitions through data?

Where are Gaussian splats genuinely useful in this kind of project, and where would they create limitations?

What should a small vertical-slice prototype prove before the full castle is produced?

What would you deliberately leave out of the first prototype?

Is this a project that a designer working with AI-assisted coding can realistically develop step by step, provided the technical architecture is chosen carefully?

My proposed first test is one polished gallery room containing three artworks, accurate wall placement, artwork interaction, a large detail view and intuitive mobile navigation.

I would be very grateful for honest feedback from people who have built interactive 3D websites, virtual galleries, museums or comparable WebGL experiences. Explanations in clear, practical language would be especially helpful, but technical terminology is absolutely welcome.

For safety and transparency, I would prefer initial technical advice to remain in this public thread. I’m not looking for unsolicited paid services, investment offers, or requests for account, repository or file access.

I can share a redacted screenshot or a minimal technical example if that would help, but the complete project documents and unpublished visual material are currently private.

Thank you

2

That’s a lot of questions! :smiley:

r.e. PlayCanvas vs. Three.js: PlayCanvas is a completely separate engine/editor platform, but if you are heavily relying on LLMs to write your codebase, Three.js is usually the safer bet. The volume of Three.js code, documentation, and community examples in LLM training data is massive compared to PlayCanvas. Pure code architectures are also much easier for coding models to manipulate directly than visual editor workflows.

r.e. Gaussian Splats vs. 3D Meshes: This comes down to navigation, collision, and rendering overhead:

  • Gaussian Splats carry no collision/geometry data natively. If you have a freely controlled WASD/FPS camera, you’d have to manually build invisible mesh collision boundaries (blockouts) around the splats. Splats are best suited for purely visual observation—like distinct museum artifacts or statues on pedestals where the user looks from the outside.

  • The Gallery Environment is far better off as standard 3D meshes built in Blender. Mesh collisions are well-understood, lightweight, and you can accelerate raycasts/shapecasts using libraries like three-mesh-bvh.

  • If you need repeated architectural elements, tell your LLM to use instancing (InstancedMesh) to keep draw calls low.

r.e. Working with LLMs: Modern models are quite good at Three.js fundamentals. Avoid over-constraining the model with rigid technical dictates if you aren’t certain of the underlying architecture; give it high-level architectural goals and instruct it to “ask clarifying questions if any implementation details are ambiguous.”
If the QA is too technical to understand, ask it to explain it to you like a noob.

3

Thank you, this is very helpful and it confirms the direction I was beginning to trust: Three.js as the foundation, with stable Blender-built mesh geometry for the castle and galleries.

May I ask three focused follow-up questions?

  1. In a hybrid setup, would you use Gaussian splats at all inside a gallery where artworks need precise wall placement and reliable occlusion, or would you keep splats only for backgrounds, atmosphere or isolated objects?
  2. For exhibitions that change regularly, would named Blender empties or nodes exported with the GLB be a robust way to define fixed artwork slots that Three.js fills from data?
  3. For one polished gallery room that must also work on iPhone and iPad, what practical performance budgets would you target for model size, draw calls, texture memory and triangle count?

Thanks again — your answer already helped me avoid another unnecessary architecture change.

und yes i used ai helper ))

4
  1. I think splats could work for environments, but my instinct is to do the environments as regular 3d meshes, and reserve splats for showing the actual exhibits if applicable. At least with meshes you have some way of navigating/colliding with the experience rather than static/fixed walk through paths.

In a gallery, you don’t always have control over the art that goes in.. it might be fixed images, or point clouds, or gaussian splats, or mesh scans… so that can be left flexible.

  1. Yup. That could work well. You could also use boxes if you want some extra info about the volumes.. but that could also be derived from the .scale of the Empty node… You can also add userData to objects in blender, that gets exported along with the glb. You do it by adding custom property name → value pairs to the object. LLM can explain that.

  2. Drawcalls < 500, the lower the better (within reason).
    Texture memory → for mobile you likely want compressed textures. The blender GLTF export has settings to enable this.. you can use webP at 75~80% quality to get good results.
    Generally you want texture sizes kept below 2k.. (the limit on lots of mobile chips)
    Triangle count.. roughly 1.5 to 2 million on mobile.. anywhere up to 16 million on high end desktops.