I built ThreeJSON — a JSON-driven declarative scene runtime for Three.js. Does this abstraction make sense?

[tianhehechu2017]

Hi everyone,

I’ve been working on an open-source project called ThreeJSON, and I recently released its first alpha version.

ThreeJSON is a JSON-driven declarative scene runtime for Three.js.

The project started from a question:

What if a Three.js scene were not primarily a body of application code, but persistent, addressable and mutable data?

In other words:

Describe 3D worlds as data, and let Three.js run them.

Try it here:

• Live demos:

  • ThreeJSON
  • Scene editor (Global, English - Work In Progress)
  • Core event mechanism (EventScript)
  • AI scene generation

• GitHub / Documentation: nnrj/threejson: ThreeJSON is a JSON-driven declarative scene runtime for Three.js, designed for persistent, mutable and extensible 3D worlds — from human-authored scenes to AI and Agent-driven generation and control.

• npm: npm install threejson

ThreeJSON is not intended to replace Three.js. Three.js remains the rendering and 3D foundation underneath it.

It is also not meant to be just a JSON wrapper around a few constructors.

The goal is to build a data-driven pipeline:

Description
→ Parsing
→ Object Creation
→ Scene Assembly
→ Runtime
→ Mutation
→ Extension
→ Persistence

Scene as Data

A ThreeJSON scene can describe not only meshes, but also cameras, renderers, controls, lights, materials, textures, external models, animations, events, resources and domain-specific objects.

A small object can be described like this:

{
  "objType": "Box",
  "position": [0, 1, 0],
  "size": [2, 2, 2]
}

Of course, saving a few lines of new THREE.Mesh() is not the interesting part.

The interesting part is that the scene now has a structured representation.

That representation can be:

• saved and loaded again;

• stored in a database;

• sent over a network;

• packaged together with resources;

• edited by tools;

• mutated incrementally;

• generated by AI;

• operated on by an Agent.

ThreeJSON currently supports both a normalized machine-oriented JSON format and a more human-friendly JSON format. The latter is normalized into the same internal pipeline before deployment.

The idea is to make scenes understandable to both humans and machines.

Runtime as Engine

A declarative scene is not very useful if the JSON is parsed once and then forgotten.

So ThreeJSON also provides a runtime layer around the resulting Three.js world.

Objects created from descriptors are registered and can be addressed through stable identities such as:

• threeJsonId

• uuid

• refName

• name

This means an application does not always need to traverse the entire scene graph or maintain its own object lookup system.

An editor, backend, business system or Agent can express operations like:

find cabinet-42
→ move it to (13, 20, 66)
→ replace its texture
→ hide it

without rebuilding the whole scene.

The runtime layer includes capabilities such as:

• object registration and lookup;

• runtime transforms and property changes;

• resource management and reuse;

• render/update scheduling;

• object and scene lifecycle hooks;

• event binding;

• animation updates;

• cleanup and disposal.

This is one of the architectural questions I would especially like feedback on:

Is there value in treating the running scene as a managed, addressable runtime, instead of only as a raw Three.js scene graph owned entirely by application code?

JSON is the canonical descriptor, but Object3D remains free

One design problem appeared very early:

If JSON is the source of truth, does that make the runtime rigid?

I did not want that.

ThreeJSON uses a two-layer model:

JSON descriptor
persistent / canonical state
        ↓
Three.js Object3D
live runtime state

The JSON descriptor can be used for persistence, synchronization, patching and reloading.

But the live Object3D is still a real Three.js object.

A physics engine can move it. AnimationMixer can animate it. A game loop can update it. Custom application code can manipulate it directly.

When the application needs to return to the data-driven flow, runtime state can be coordinated back with the descriptor.

The goal is not to build an abstraction that fights against Three.js.

The goal is to preserve the freedom of the Three.js runtime while adding a persistent and machine-operable representation above it.

Objects can have behavior, not just appearance

ThreeJSON can describe more than geometry and materials.

Objects can also have:

• declarative animations;

• model-embedded animations;

• mouse and keyboard events;

• built-in actions;

• EventScript logic;

• custom JavaScript behavior;

• runtime commands.

So scene data can gradually represent:

Object + State + Animation + Event + Behavior

For example, an object can rotate continuously, react to a double click, trigger an action, execute a script, or be changed later through the runtime API.

Domain as Extension

I don’t think every higher-level 3D concept should become another primitive object type in the core.

A cabinet is not a primitive. A cargo ship is not a primitive. A robot, lab instrument, factory device or data-center rack should not have to be hard-coded into the engine.

So ThreeJSON has a Domain system.

A domain can define its own:

• model definitions;

• JSON structures;

• handlers;

• APIs;

• behaviors;

• visual conventions.

For example, the current project includes domain-oriented work around cabinet/data-center scenes and port scenes.

The core runtime does not need to know what a cabinet or cargo ship means. It only provides the mechanism through which that domain participates in scene creation and runtime behavior.

The intended structure is:

ThreeJSON Core
+
built-in objects
+
your own domain model system
+
your own runtime behavior
+
third-party extensions

This is why one of the principles of the project is:

Domain as Extension

Third-party systems should remain third-party systems

The same principle applies to physics engines and other large external capabilities.

ThreeJSON is not trying to reimplement everything.

The project provides extension mechanisms, lifecycle hooks and frame hooks so optional systems can participate in the runtime.

A physics engine should remain a physics engine. ThreeJSON’s job is to provide a clean place for it to connect to a data-described scene and its lifecycle.

Users should also be able to create their own object types, controls, registries and domain systems instead of being limited to what the project already provides.

Agent as Operator

This is the part that made me continue the project even after AI became very good at writing Three.js code.

At one point I asked myself:

Why build this at all? Why not just ask AI to generate native Three.js code?

My current answer is:

Generating code and operating a world are different problems.

Imagine an AI has generated a large Three.js application.

Later, the user says:

“Move the third cabinet in the second row 1.5 meters to the left and open its front door.”

With source-code generation, the AI may need to inspect the project, understand object relationships, find the right variables, modify code and hope the change does not break something else.

In a structured world, the flow can instead be:

User request
→ AI inspects scene/object information
→ AI resolves the target object
→ AI generates a small command
→ ThreeJSON parses the command
→ runtime mutation API performs the change

Conceptually:

find cabinet-03
→ translate x by -1.5
→ find front-door
→ rotate to open angle

The AI does not need to regenerate the world.

It only describes the change.

ThreeJSON explores several levels of AI / Agent interaction.

JSON Patch

If only one part of a descriptor needs to change, an Agent can produce a small JSON Patch instead of regenerating the full scene.

For example:

[
  {
    "op": "replace",
    "path": "/position/0",
    "value": 12.5
  }
]

This can update the canonical descriptor and participate in the runtime synchronization flow.

Runtime commands

For operations that are naturally imperative, AI can generate compact command sequences.

The intended flow is:

natural-language request
→ AI-generated command
→ ThreeJSON command parser
→ runtime mutation API
→ live scene changes

This is useful for operations such as:

select object A
move it beside object B
rotate it 90 degrees
replace its material
hide objects matching a condition

Spatial context

A useful Agent needs more than object names.

The scene can expose structured object and spatial information so AI can reason about positions, bounds and nearby objects rather than guessing from source code.

This makes requests such as:

“Place this machine between A and B without overlapping the wall.”

a structured scene-operation problem, not a code-editing problem.

The goal is not to claim that AI can already solve every 3D task perfectly.

The goal is to give AI a smaller, more precise and safer action surface than “rewrite the Three.js application.”

Editor, player and examples

The repository is not only the core library.

It currently includes:

• a scene editor;

• a scene player;

• room / cabinet examples;

• port scene examples;

• many standalone HTML demos and tutorials;

• runtime mutation examples;

• event and script examples;

• domain examples;

• import/export related experiments.

The editor is useful both as a scene-building tool and as a place to inspect and test runtime capabilities.

The player provides a lighter way to load and run scenes.

The project also explores packaging scenes with resources and importing/exporting external 3D models, so that ThreeJSON does not become an isolated ecosystem.

What could this be useful for?

Some possible use cases I am interested in:

• digital twins;

• data centers, factories, ports, warehouses and campuses;

• AI-assisted 3D scene generation and editing;

• procedural game map generation;

• scientific simulation and visualization;

• robot training environments;

• structured generation of simple 3D-printable models;

• browser-based 3D editors;

• Agent-operated 3D worlds.

Some of these are already close to the current direction of the project. Others are still hypotheses that need real experiments and feedback.

The philosophy

I eventually condensed the project into four lines:

Scene as Data — The scene is data.
Runtime as Engine — The runtime is the engine.
Domain as Extension — Domains are extensions.
Agent as Operator — Agents are operators.

Or, in plain English:

You define the world. AI creates and reshapes it. ThreeJSON brings it to life.

The project is still alpha.

I am not posting this because I think the architecture is finished. I am posting it because I would like people who actually build things with Three.js to challenge it.

I would especially appreciate feedback on these questions:

• Does this abstraction solve a real problem, or is it too much structure on top of Three.js?

• Where does ThreeJSON fight against Three.js instead of helping it?

• Is the canonical-descriptor / live-Object3D split understandable?

• Does the Domain system make sense?

• Is the AI command / JSON Patch approach useful, or would you design the Agent interface differently?

• Which part would you remove if you wanted a much smaller core?

Critical feedback is very welcome.

Thanks for taking a look.

[manthrax]

This is interesting.

Have you used the threejs editor + it’s save function?

three.js editor

It saves as .json and allows things like event handling/scripting etc.

I wonder if there is some middle ground that could leverage more of the infrastructure provided by threejs already…

Cool stuff! Looking forward to where you take this..

[tianhehechu2017]

Thanks for the reply! And yes — I have used the official Three.js editor, and I’m also familiar with Three.js’s built-in JSON serialization through scene.toJSON().

I think ThreeJSON has a different scope and purpose, though. The difference becomes clearer if I separate the JSON model from the editor.

First, the JSON model

As I understand it, the JSON produced by Three.js’s scene.toJSON() is primarily a serialized description of a scene.

ThreeJSON uses JSON as something broader. In addition to describing the static scene, it can also describe object behavior, interaction, and domain-specific models.

More importantly, ThreeJSON is intended to support the full lifecycle of a declarative 3D scene.

For example, it provides:

• Object registry and identity — every object declared through JSON can receive a stable threeJsonId, so it can be retrieved and operated on later at runtime.

• Lifecycle hooks — operations can be injected before/after scene or object loading, and before/after disposal.

• Events and scripting — events and actions can be declared directly in JSON. For example: when Box A is double-clicked, move it 5 units to the left, change its color to blue, and hide Sphere B.

• Domain encapsulation — higher-level concepts can be defined as domains rather than being hard-coded into the core. For example, a data-center domain can define cabinets and devices, while another domain can define completely different business objects and behaviors.

• AI interfaces — AI can generate complete scenes or objects through JSON, but it can also make small runtime changes by generating commands such as:

 move box_a position x - 2

ThreeJSON parses the command and maps it to the runtime mutation API, so the AI does not need to regenerate or rewrite the whole scene.

• .tjz packages — essentially ZIP archives that package the JSON together with textures and other resources for import/export.

There are many other capabilities documented in the doc directory, but I think the more important distinction is this:

JSON is the canonical source of truth for the ThreeJSON interpreter, but JSON itself is only one of the interfaces through which ThreeJSON can be operated.

Currently, ThreeJSON can be operated through:

• JSON

• commands

• AI-driven natural language

• the built-in DSL

• JavaScript APIs

• the editor UI

Internally, these interfaces eventually converge on the same descriptor/runtime system.

So, in a sense, ThreeJSON’s JSON is closer to a declarative, programmable scene language expressed in JSON syntax — partly because that representation is also friendly to AI and other machine-generated workflows.

That is quite different from using JSON primarily as a serialized snapshot of a scene.

Then, the editor

The official Three.js editor is mainly a general-purpose tool for visually creating, editing, importing and exporting Three.js scenes.

The ThreeJSON editor has a different role: it is a graphical interface over the broader ThreeJSON runtime.

So the editor is not the foundation of the project. It is one of several interfaces for using the same underlying capabilities — object management, runtime mutation, events, scripts, domains, AI operations, import/export, and so on.

You can try the current full editor here:

ThreeJSON Editor:

场景编辑器

The full version currently only supports Chinese. I’m in the process of migrating the UI to a multilingual system, but that work is not finished yet.

That said, I think your point about finding a middle ground and leveraging more of the infrastructure already provided by Three.js is a very good one.

ThreeJSON should not reimplement things that Three.js already does well. One of the questions I’m still thinking about is exactly where ThreeJSON should build its own abstractions, and where it should integrate more directly with existing Three.js infrastructure.

Thanks again for taking the time to look at the project and reply — I really appreciate it!

[manthrax]

Right. which is kinda why I asked about it:

looking at this from a native Three.js perspective, I have a few thoughts and sanity checks on where this fits compared to existing patterns:

• Object Registry & Identity:

You mentioned using threeJsonId for stable tracking. Just as a note, the native Three.js JSON representation already handles this out of the box using stable UUIDs for persistence and runtime-unique integer .ids for performance.

• Lifecycle Hooks & Scripts:

The native Three.js Editor already accomplishes a version of this. You can wire scripts directly to events like init, play, and stop, and those scripts persist seamlessly within the serialized JSON scene graph.

Domain Encapsulation & AI Interfaces

I think the Domain system is where your project gets really interesting and brings unique value. However, I have a slight reservation regarding the AI layer:

While LLMs are incredibly skilled at writing native Three.js + JavaScript code, they might struggle with an indirect, abstract schema. If I want an LLM to build a scene, I can leverage its deep training data on vanilla Three.js. If I force it to use a custom schema, I have to dump that entire schema definition into the prompt context window.

While text-to-code via JavaScript can be messy, it requires zero context overhead for the LLM. For example, commands like move box_a position x - 2 are already trivially generated by an LLM writing native JS.

An Alternative Mental Model: A 3D MCP? I don’t think ThreeJSON is a bad idea at all—I just wonder if it could be moved closer to the existing Three.js engine/editor patterns rather than inventing a parallel runtime.

In fact, reading your section on the Agent interface, it sounds less like a new file format and more like a Model Context Protocol (MCP) server for Three.js.

Instead of treating ThreeJSON as a standalone runtime wrapper, what if it functioned as an abstraction layer that exposes native Three.js scenes to LLMs via standardized tools (e.g., exposing a locate_object, patch_property, or trigger_animation tool to an Agent)? That way, the LLM doesn’t have to learn a new JSON schema—it just calls tools to manipulate a standard Three.js world.

Really interesting work, looking forward to seeing how this evolves!

[tianhehechu2017]

Thank you for the thoughtful reply and suggestions.

Yes — the long-term goal of ThreeJSON is indeed to act as an intermediate layer between AI and Three.js. Your description of a “3D MCP” is actually very close to an important part of what I am trying to build.

Let me clarify a few of the design choices.

Object registry and identity

ThreeJSON’s object manager uses threeJsonId as its primary identifier, while remaining compatible with Three.js uuid values and refName aliases.

There are several reasons why I introduced threeJsonId instead of relying exclusively on uuid.

First, UUIDs are not very human-readable. A threeJsonId can be explicitly assigned by a user or AI, as long as it is unique. This makes references such as:

 cabinet-42
 front-door
 temperature-panel

much easier to reason about than opaque UUIDs.

Second, the ThreeJSON registry does not only manage Object3D instances. It also tracks other scene entities and resources, some of which are not native Three.js objects and therefore do not naturally have Three.js UUIDs — for example information panels, audio resources, external models, native Three.js JSON sub-scenes, and Domain-defined models.

Third, the interpreter sometimes needs to identify, preprocess and track descriptors before the corresponding Three.js objects exist.

For example, a user may define an information panel separately in an infoPanelList, assign it a threeJsonId, and then reference it from a box’s subScene. The panel could of course be embedded directly inside the box descriptor, but separating reusable objects often makes larger JSON scenes much easier to read and maintain.

So threeJsonId is intended to identify the declarative entity, not merely the resulting Three.js runtime instance.

That said, threeJsonId is optional. If the user does not provide one, ThreeJSON can automatically fill it using a UUID.

I think the distinction is roughly:

 threeJsonId -> declarative / semantic entity identity
 uuid        -> native Three.js object identity
 refName     -> human-friendly alias
 name        -> sharing/batch operations of similar objects
 label       -> any UI name

Your comment is still a useful reminder that ThreeJSON should reuse native Three.js identity mechanisms wherever their semantics already match the problem, rather than duplicating them unnecessarily.

Lifecycle hooks and scripts

When I implemented this part of ThreeJSON, I was aware that Three.js and the Editor already had mechanisms that could potentially be reused.

For performance and architectural reasons, I currently reuse some of the native mechanisms at the scene level, but the lifecycle of individual declarative objects does not deeply hook into the internals of Three.js object creation.

Your point is a good one, though. I will look more carefully at how the existing init, play, and stop script/event model could be reused or integrated rather than building a completely parallel mechanism.

Domain encapsulation

I’m glad you noticed the Domain system, because it is an important part of the architecture.

A Domain allows users to encapsulate models from their own problem space, including not only the model structure, but also its behavior and actions.

More fundamentally, Domains can extend the JSON vocabulary understood by ThreeJSON.

A Domain can register its own sub-parser, after which users can describe objects in the language of their own field. Conceptually, something like:

 {
   objType: "domain",
   domain: "room",
   door: {
     type: "double"
   },
   window: {
     type: "glass",
     color: "blue"
   }
 }

The core interpreter does not need to understand what a “double door” means. The room Domain does.

The same Domain can also expose APIs to the host application, so application code can interact with the higher-level model rather than manually manipulating all of its underlying meshes.

This is one reason I think the Agent layer may eventually be more useful at the semantic level than at the raw Three.js level. An Agent should ideally be able to operate on concepts such as a cabinet, device, door, or room, rather than always reasoning about arbitrary groups of meshes.

AI interfaces and schema overhead

I agree with your concern here.

An LLM already knows a great deal about vanilla Three.js and JavaScript. A custom schema has a real context cost, and forcing a model to learn a large schema just to do:

 box.position.x -= 2;

would obviously not be an improvement.

The command example I gave earlier was too simple to demonstrate the actual reason for the AI layer.

ThreeJSON provides AI-oriented skills and structured interfaces, but the goal is not simply to replace JavaScript syntax with another syntax.

I see several advantages to the data-oriented approach.

First, a JSON-described scene is directly persistable. It can be stored in a database, transmitted over a network, packaged with resources, or exported through Three.js / ThreeJSON workflows to other formats. The textual representation is also relatively compact.

Second, the main schema/context cost is concentrated around scene generation and understanding the available operations. Later changes do not necessarily require the Agent to understand the entire source-code architecture again. It can receive only the relevant scene, object, Domain, and spatial context, then perform a targeted operation against a stable threeJsonId.

Third, structured output can be validated and constrained more precisely than arbitrary generated JavaScript.

And fourth, the scene remains data-driven rather than being hard-coded into application logic. ThreeJSON supports patch-style updates, including JSON Patch-style incremental changes, which are useful for runtime updates and network synchronization.

So I think the more accurate model is not:

 LLM
 -> learn the entire ThreeJSON schema
 -> generate everything as custom JSON forever

but rather:

 LLM / Agent
 -> receive only the relevant semantic and spatial context
 -> call a constrained tool or generate a small patch/command
 -> ThreeJSON resolves the target
 -> runtime API mutates the live Three.js world

In that sense, I agree strongly with your MCP interpretation.

Tools such as:

 locate_object
 patch_property
 trigger_animation

are very close to the direction I want the Agent interface to take.

The important thing is that the Agent should not need to understand the entire scene or rewrite the application. It should receive the relevant context and operate through a small, controlled interface.

Closer integration with Three.js

I also agree that ThreeJSON should stay as close as possible to the existing Three.js engine and reuse its infrastructure wherever that makes sense.

ThreeJSON is completely built on top of Three.js, and most of its actual 3D capabilities come directly from Three.js. But you are right that there are places where the current implementation does not yet reuse existing Three.js mechanisms as much as it could. I would like to improve that over time.

The editor may also have caused some confusion about the project’s architecture.

The editor is not the core product of ThreeJSON. The main project is essentially:

 Three.js + ThreeJSON core + optional Domains + optional extensions and tools

In fact, the npm package is centered mainly around the core and Domain capabilities, and even the built-in Domains are optional examples intended to help users understand the extension model.

The editor included in the repository is closer to a visual interface and demonstration environment for the core runtime. It helps developers explore the available capabilities, test JSON descriptors, and imagine how those capabilities could be integrated into their own applications.

So yes — from the Agent perspective, describing ThreeJSON as a kind of MCP-style semantic and operational layer for Three.js is very close to the direction I have in mind.

Your comment also made me realize that I may have over-emphasized “AI generates ThreeJSON JSON” when presenting the project, and under-emphasized the more important idea:

ThreeJSON can expose a structured, semantic, spatially aware, and constrained 3D world to Agents.

The JSON format, runtime, Domains, commands, patches, and Agent tools are different interfaces around that larger idea.

Thank you again for taking the time to think through the architecture and give such detailed feedback. This is exactly the kind of discussion I was hoping for when I posted the project.

And of course, contributions and further criticism are very welcome!

[PavelBoytchev]

Here are few comments:

1. Compatibility
As Three.js is rapidly evolving, how do you cover deprecated or emerging features? If you take a JSON file, how would it know which Three.js release is compatible with it?

2. Uniqueness
Some things look like having several optional definitions, e.g. OrbitControls could be defined as:

controls: {
  enableDamping: true,
  ...
},

or as:

{
  "objType": "controls",
  "type": "orbit",
  enableDamping: true,
  ...
}

3. Procedural
I’m also concerned with the general procedural creation of scenes which play a vital role in more complex models. I could not see how this fits in ThreeJSON. One type of procedural code is when the buffers are created and filled by code; another type is when the standard objects are used (e.g. a sphere) and then they are procedurally modified.

4. Addons
Some addons are well incorporated (like controls), but what about other addons (e.g. SimplexNoise)?

5. Mapping
It might be good if there is an easy to understand mapping, because some of the terminology in ThreeJSON differs from that of Three.js. Maybe a taxonomy or even ontology might be also useful.

6. Converters
Maybe you already have it, but it would be nice to have exporter from current in-memory scene to ThreeJSON, and to include only those properties, which are not with default values.

7. Impression
My personal impression is that you started developing this from examples toward Three.js core. This is, you cover what is practically used often. A side effect of this could be that some feature of Three.js may remain completely out of sight, and thus, unsupported in ThreeJSON.

8. Comments
JSON does not supports comments. Of course, there are some tricks how to circumvent this by either relaxing the syntax, or expanding it, or using fake properties… But if the definition of something is supposed to be read by people, then comments are essential.

9. Demos
Yes, you have a lot of demos and they demonstrate nicely many features of ThreeJSON. Have considered any demos focused on the benefits of ThreeJSON compared to raw JS code? I’m not talking about toy demos, but something more complex.

10. TSL
Any plans to have JSON support for entities defined in TSL?

[tianhehechu2017]

Thank you for joining the discussion. You have raised many important questions at the architectural level. I’ll try to answer them one by one.

1. Compatibility

ThreeJSON currently uses three mechanisms to maintain compatibility.

First, compatibility at the JSON level.

A ThreeJSON document can contain a version field. This tells the interpreter which version of the ThreeJSON format the document was written for.

Second, compatibility inside the ThreeJSON core.

There is a dedicated compatibility layer under /core/compat. It is responsible for both:

• compatibility across different Three.js releases; and

• compatibility with older ThreeJSON documents according to the version specified in the JSON.

The current version of ThreeJSON officially supports Three.js releases r179–r184.

Through compatibility adapters, ThreeJSON guarantees consistent behavior down to r179. In testing, it can remain largely compatible as far back as r153, although identical behavior is not guaranteed for those older releases.

There is a document about this at /doc/three-compat.md, but unfortunately it is currently only available in Chinese. I will add an English version as soon as possible. I am also considering making English the default language for documentation in future releases.

Third, long-term API compatibility.

My current policy is that APIs in long-term-supported versions will never simply disappear. Even if an old API contains a bug, the old behavior may be preserved as a legacy versioned behavior rather than silently changed.

If the correct version is specified, the goal is for the API behavior to remain consistent with that historical version.

These three mechanisms are intended to address not only changes in Three.js itself, but also changes in ThreeJSON, so that upgrading the runtime does not invalidate users’ existing scene assets.

2. Uniqueness

The two ways of defining OrbitControls are intentional.

The reason is to reduce the conceptual burden for different kinds of users.

For most users, OrbitControls is part of the scene infrastructure. They think of it as a global runtime component, so ThreeJSON allows a convenient top-level form such as:

 controls: {
   enableDamping: true
 }

However, Three.js itself allows multiple OrbitControls instances to exist in a scene, as long as the application manages which one is active to avoid conflicts.

In some applications — for example, camera switching in an FPS-style application — a user may need multiple controls. In that case, controls are better treated as individual runtime objects, more like a box or another objType:

 {
   objType: "controls",
   type: "orbit",
   enableDamping: true
 }

If both forms are used, the interpreter normalizes and merges them into the scene rather than treating them as mutually exclusive definitions.

There are a number of similar cases in ThreeJSON where the same capability can be introduced through both a convenient high-level configuration and a more explicit object-level descriptor. Depending on the feature, either the global configuration or the local definition may take precedence.

That said, your question makes me realize that these rules need to be documented much more clearly. The distinction between multiple input forms and the normalized internal representation should be explicit, especially for users, schema validation, AI generation, and tooling.

3. Procedural creation

I understand your concern. Procedural scene and geometry generation are essential for complex 3D applications.

ThreeJSON is intentionally non-invasive.

For an LLM, ThreeJSON can act as an intermediate layer. For an application, it acts as an interpreter and runtime. For a human developer, I intend it to remain an optional, non-invasive framework.

Suppose an existing Three.js application is already written entirely in JavaScript. Introducing ThreeJSON does not require rewriting that application or moving all scene creation into JSON.

A developer can decide how much of ThreeJSON to use.

For lightweight usage, it is possible to import only APIs from the core. Conceptually:

 import { deployMesh } from "threejson/core";

Then, inside an existing code-driven Three.js application, a developer could receive or define a sphere descriptor and deploy only that object:

 deployMesh(sphereJson, scene);

The resulting sphere is added to the existing Three.js scene and registered with the ThreeJSON object manager.

If the descriptor contains a threeJsonId, refName, or name, the application can later retrieve the resulting native Three.js object:

 const sphereMesh = getObjectByThreeJsonId(sphereThreeJsonId);

From that point onward, it is still a normal Three.js object. The developer can procedurally modify its geometry, buffers, material, position, or anything else using ordinary JavaScript and Three.js APIs.

So the intended model is not:

 JSON replaces procedural code

but rather:

 existing Three.js code
 +
 optional ThreeJSON descriptors
 +
 optional ThreeJSON runtime services

Procedural code remains procedural code. ThreeJSON can participate where structured description, registration, persistence, runtime lookup, or AI operation is useful.

4. Addons

ThreeJSON has several ways to deal with Three.js classes or addons that do not yet have dedicated built-in adapters.

1. Native type inference

ThreeJSON supports native type inference.

The intention is that, in many cases, a user can try describing a Three.js or addon type even if ThreeJSON does not have a dedicated hand-written adapter for it.

Conceptually, something like:

 {
   objType: "simplexNoise",
   threeJsonId: "noise2D",
   noise: {
     x: 0.1,
     y: 0.2
   }
 }

The inference mechanism can inspect the available type and descriptor and attempt to construct or adapt it automatically.

Of course, I do not want to claim that every arbitrary addon can always be inferred correctly. Addons with unusual constructors, procedural APIs, closures, callbacks, or non-serializable state may require explicit integration.

2. Native JSON sub-scenes

ThreeJSON supports nativeJson, so native Three.js objects or sub-scenes that can be serialized through the existing Three.js JSON infrastructure can be embedded and loaded inside a ThreeJSON scene.

3. Create it in JavaScript and register it

A developer can create an unsupported object normally:

 const simplex = new SimplexNoise();

and then register it with the ThreeJSON object manager through registerObject.

It can then participate in ThreeJSON’s identity, lookup, and management mechanisms without requiring ThreeJSON to own its creation.

4. Add a dedicated adapter

If none of the above approaches is sufficient, a dedicated adapter can be added.

So I need to distinguish more clearly between:

what ThreeJSON supports through built-in adapters

and:

the actual capability boundary of ThreeJSON

The latter should not be limited to the Three.js features I have manually wrapped.

5. Mapping

I think your point about terminology is very important.

The current documentation still needs further review and normalization. Where possible, I would like ThreeJSON terminology to align more closely with Three.js terminology.

For concepts that intentionally differ, I agree that a clear mapping document would be useful.

For example:

 Three.js concept
 ↕
 ThreeJSON descriptor / runtime concept
 ↕
 Domain-level concept

A taxonomy — and perhaps eventually something closer to an ontology for semantic and Agent-oriented use cases — could also be valuable.

Thank you for this suggestion. I think it is an important one.

6. Converters

The ThreeJSON core already provides APIs for extracting ThreeJSON descriptions from a scene.

The import/export functionality in the online scene editor is built on top of this capability.

ThreeJSON also benefits directly from the large import/export ecosystem already available in Three.js.

The general conversion path can be:

 external model
 → Three.js scene
 → ThreeJSON

and, where supported by the relevant exporter:

 ThreeJSON
 → Three.js scene
 → external format

Your suggestion about exporting only properties that differ from their default values is particularly interesting. This would make generated ThreeJSON smaller, easier for humans to read, and cheaper for AI systems to process.

That is something I would like to improve further.

7. Impression

Your impression points to a real issue.

There are still many Three.js capabilities that ThreeJSON does not support through dedicated built-in descriptors or adapters, and I intend to add more over time.

However, because ThreeJSON is non-invasive, a project that introduces ThreeJSON still retains access to the full Three.js API. Unsupported features do not have to wait for ThreeJSON before they can be used.

I can understand why the current project gives the impression of developing “from examples toward the Three.js core.” The current demo structure probably contributes to that impression.

I am currently rebuilding the demo system in a style inspired by the ECharts examples: JSON or editable definitions on one side, and the live 3D result on the other.

The goal is to make the data-driven nature of ThreeJSON immediately visible, rather than presenting a collection of isolated examples that may look like individual Three.js features being reimplemented one by one.

8. Comments

I have thought about this problem, and I agree that it is a real limitation.

AI does not necessarily need comments in scene data, but humans do.

I am considering supporting an extended JSON input format that allows comments, with the interpreter stripping or normalizing them before the document enters the canonical parsing pipeline.

I am also considering supporting a conventional remark field for semantic notes that should remain part of the data itself.

The important distinction may be:

 authoring format
 → may contain comments and human-oriented conveniences
 ​
 canonical JSON
 → strict, portable, machine-oriented representation

I have not finalized this design yet, but I agree that human-readable scene definitions need a better answer than “JSON does not support comments.”

9. Demos

This is an excellent point, and it is exactly the problem I am working on now.

Several users have told me that the current online demos are confusing at first: they can see the rendered result, but they cannot immediately see where the JSON is or why ThreeJSON is useful.

The most common suggestion has been to create something similar to the ECharts example pages:

 editable JSON / code
 │
 │ live update
 ▼
 3D canvas

I am currently implementing and debugging a new demo system based on this idea. I expect to publish it this week and use it to replace the current main demo experience.

Most of the existing demos intentionally use minimal scenes because each demo was designed to explain one specific feature. I assumed that developers or AI systems might use those pages as focused references, so I kept the scenes simple and tried to isolate the feature being demonstrated.

In retrospect, I think I optimized too much for reference documentation and too little for first impressions.

The current demos often prove that:

“ThreeJSON can do this.”

But they do not yet clearly prove:

“This is why I would use ThreeJSON instead of writing the same complex system directly in raw JavaScript.”

I agree that a more complex, comparative demo is needed, and the new demo system is intended to move in that direction.

10. TSL

I do not yet have a finalized JSON representation for TSL, but I do consider it an important direction.

TSL is particularly interesting because it is not simply another serializable Three.js object type. It is a compositional, programmable system, so I do not want to rush into creating a second, JSON-shaped version of the entire TSL API.

I think there are several possible levels of integration:

1. Reference registered TSL entities from ThreeJSON — TSL graphs or functions remain native code, while ThreeJSON descriptors refer to them by stable identifiers.

2. Expose selected TSL structures declaratively — common node graphs could be represented as structured data where this is genuinely useful.

3. Allow Domains or extensions to provide their own TSL-aware parsers — keeping TSL support outside the core when it is domain-specific.

4. Preserve native TSL access — because ThreeJSON is non-invasive, applications should always remain free to create and modify TSL entities directly in JavaScript.

My current instinct is to begin with registration, references, and extension points, rather than trying to serialize the entire TSL programming model into JSON.

However, I have not finalized the design, and I would be very interested to hear what kind of TSL support you had in mind when you asked the question.

Thank you again for taking the time to look through the project and raise these questions. This kind of architectural feedback is extremely useful to me.

[PavelBoytchev]

Thanks for the very detailed answer.

@tianhehechu2017

However, I have not finalized the design, and I would be very interested to hear what kind of TSL support you had in mind when you asked the question.

Maybe “TSL” was not the best word. In Three.js there is a node system which is tightly coupled with TSL. It allows to use TSL to define custom nodes that may represent materials, properties, geometries, compute shaders, and so on. So, you may have an object with material properties defined in TSL (e.g. the color is not a color value, but a node defined via TSL expression).

For example, see the TSL elements in the material in this demo:

https://codepen.io/boytchev/pen/bNeNObV

And for the span of the node system in Three.js, here are two snapshots from The Three Tree project - the whole Three.js and a closeup. The red bulbs are node-related entities (classes, functions, etc). This is just to get a feeling of how much is now related to nodes.