Using ChatGPT AI and Three.JS to Help Code 3D Games

5 Likes

This is like watching a horror movie, this is the end of us … we are screwed :sweat_smile:

Its very helpful with generating simple boiler plate code samples. Code is 80 to 90 percent usable. Sometimes it uses threejs coding style from 5 years ago. When asking to generate web xr sample code, it makes a lot of assumptions. Also, asking it to draw a specific shape it usually just ends up drawing a triangle.

However, generally, its great at helping to discover new ways of doing things.

I’ve found the best way to start is have it describe 10 points about the concept you want to build. Then ask it to expand on each point and generate the code from that. Generally get better results this way. Don’t forget to ask it about any limitations about your choices. That’s always an interesting read.

Here’s an interesting prompt that yields a very good result “imagine and describe the architecture for a threejs whiteboard component”. Then ask it to “add code to change the color of the pen” or “add code for a pen to erase” and “add code to undo last draw operation”

I’ve been using ChatGPT to help write the code for a threejs scripting engine. I have a basic interpreter working. The AST can be translated to JavaScript to hopefully load as a runtime script block for faster execution.

I’m thinking of creating a domain specific language using PEG.js to make scripting easier. Hopefully, ChatGPT can help with that.

4 Likes

That sounds like a cool project. Using AI to help you code is a great way to help you increase lead times on projects!

:rofl: At first it seems scary but then learn to use AI to your advantage!

2 Likes

@Fennec three.min.js :man_facepalming::rofl::rofl: this AI is trained with r87, a GAN and some training references to r148 would be needed before chatgtp could really generate cool stuff and potentially be a worry! @Shane_Brumback @anidivr agree fully that AI is very helpful in computing simple boiler plate scenes, it’s really helpful, I could imagine that people unbeknownst to three.js and code in general could learn a whole lot from this type of tech on demand. Imagine having a full on conversation with an AI where a project is based on what’s previously been achieved, you could really pictorially describe the types of textures and materials required for a certain object with choices as to how to generate said texture / material

2 Likes

Yah that would be sweet! I will also try to tell it to just use the latest version regardless and see what I get. I had it attempting to add custom textures with PNG this morning. I will also see if I can get it to generate some volumetric spotlights like in this game that I worked on. The most benefit I am seeing up until now is the speed at which I can prototype and test ideas.

2 Likes

@Shane_Brumback @Lawrence3DPK, I know I’m being a drama queen here :sweat_smile:, but you gotta admit, this thing and its potential evolution will be the end of us.

On a more serious note, whether I like it or not doesn’t matter, this is progress, it’s adapt or fall behind, and I think we are at a sweet point here, where it is just smart enough to be used as a great tool and a significant productivity boost, I’ll give it a try.

@Shane_Brumback , I would love to see it generate some kind of particle system, I’ve seen some videos where it does it with blender and it was (scary) amazing.

1 Like

This was actually one of the first things I tried with it and it seemed to work pretty well, it did a great job constructing the most simple scene setup with 100 animated points using buffer geometry positions and points material, the method it used for updating the points positions needed a bit of tweaking but was really close to a sweeping dust cloud, really neat.

1 Like

can you share the prompt you asked ChatGPT to generate particles using threejs?

Write three.js code that animates 100 points

Yeilds…

// First, we create a scene and add a camera and a light
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
const light = new THREE.DirectionalLight(0xffffff, 1);
scene.add(camera);
scene.add(light);

// Next, we create an array of 100 points and add them to the scene
const points = [];
for (let i = 0; i < 100; i++) {
  const pointGeometry = new THREE.SphereGeometry(0.05, 32, 32);
  const pointMaterial = new THREE.MeshBasicMaterial({ color: 0xff0000 });
  const point = new THREE.Mesh(pointGeometry, pointMaterial);
  point.position.set(Math.random() - 0.5, Math.random() - 0.5, Math.random() - 0.5).normalize();
  point.position.multiplyScalar(Math.random() * 10 + 1);
  points.push(point);
  scene.add(point);
}

// Then, we create a renderer and append it to the DOM
const renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);

// Finally, we create a function to animate the points
function animate() {
  requestAnimationFrame(animate);
  for (const point of points) {
    point.rotation.x += 0.01;
    point.rotation.y += 0.01;
  }
  renderer.render(scene, camera);
}

// Start the animation loop
animate();

As I say the animation loop is pretty useless but the prompt is very basic, I think I used a bit more description previously

EDIT: as assumed, even a slighlty more detailed prompt yeilds a better way to create the animated points… yet as you can see it’s using the Geometry class from before r125’s update to BufferGeometry

three.js code that animates cloud of 100 points with points material

yeilds…

// Set up the scene, camera, and renderer
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
const renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);

// Create a geometry with 100 randomly positioned vertices
const geometry = new THREE.Geometry();
for (let i = 0; i < 100; i++) {
  const vertex = new THREE.Vector3();
  vertex.x = Math.random() * 20 - 10;
  vertex.y = Math.random() * 20 - 10;
  vertex.z = Math.random() * 20 - 10;
  geometry.vertices.push(vertex);
}

// Create a points material and set its size
const material = new THREE.PointsMaterial({ size: 0.5 });

// Create a points object and add it to the scene
const points = new THREE.Points(geometry, material);
scene.add(points);

// Animate the points by updating their positions each frame
function animate() {
  requestAnimationFrame(animate);

  // Update the position of each vertex
  for (let i = 0; i < 100; i++) {
    const vertex = geometry.vertices[i];
    vertex.x += Math.random() * 0.1 - 0.05;
    vertex.y += Math.random() * 0.1 - 0.05;
    vertex.z += Math.random() * 0.1 - 0.05;
  }

  // Tell three.js that the geometry has changed and needs to be re-rendered
  geometry.verticesNeedUpdate = true;

  renderer.render(scene, camera);
}

animate();
2 Likes

That’s impressive, at this point, I wouldn’t be surprised if it can generate some complex Shaders as well. Since I’m not very comfortable with GLSL, this can be quite helpful.

Unfortunately I can’t run it from my country “Algeria”, same thing for China, Russia, Egypt, Ukraine… I can bypass the restrictions but it’s a bit tricky, I’ll give it a shot while I’m procrastinating.

2 Likes

That’s the worst bad practice code i could imagine :sweat_smile:

3 Likes

yeah in terms of optimal usage and performance it’s aweful, in terms of a basic quick reference to how something like this would be done i guess it’s ok, of course i agree with you fully, geometries and materials should definitely be reused and or cloned, the updated edit ^^^ is a lot more friendly in terms of practicality for sure, bare in mind these generated examples would essentially be really rough starting guides you could modify to make suit best practices…

1 Like

For one thing, I needed a regular expression. I had not done that for a long time. The solution from chatGPT delivered was what I wanted.

Then I tried to use chatGPT to help with this example.
Create circle with fuzzy edge made of individual random particles - #10 by hofk => SphereWithRandomPointsl
That went pretty wrong.

As a first solution I was offered to set the points on the intersections of longitudes and latitudes. This is not random but even distribution. The code itself was suitable for beginners.

I objectively criticized the result and got a “You’re right …”

The next proposal was interesting but far from the requirement.

It was a Spiral - Wikipedia : Spherical spiral on which the points were evenly spaced.

Upon further inquiry, chatGPT behaved almost humanely. Like an expert who is at a loss, but gives many general hints. Some of them off-target to disguise the fact that he doesn’t know everything either.

I found the solution in a German mathematics forum that has nothing to do with programming. It was about the pure mathematical problem.
There it becomes clear that only what was present in the training material can appear in solutions.

In the future, the system should also be able to access content online at some point. Then it will look different again.

2 Likes

Very interesting, I think results of course are heavily limited to the training material fed to it, although asking it to write a 1000 word essay for instance on any topic I’ve tested so far seems to suggest it may have some limited access to the Internet, I wonder who it’ll be to fork the repo and add training modules based on the latest versions of 3js :thinking:

1 Like

Nice work guys! :rofl: :rofl: Sorry to be the one to open the can of worms…I am currently trying to get it to generate 3D gltf JSON then import to the ThreeJS editor. The Ai keeps crashing on a 3D JSON chair asset but it’s attempting and doing a pretty good job so far.

{
“asset”: {
“version”: “2.0”,
“generator”: “3D modeling software”
},
“scenes”: [
{
“name”: “Scene”,
“nodes”: [
1
]
}
],
“nodes”: [
{
“name”: “Chair”,
“mesh”: 0,
“children”: [
2,
3,
4,
5
]
},
{
“name”: “Leg_Front_Left”,
“mesh”: 1
},
{
“name”: “Leg_Front_Right”,
“mesh”: 1
},
{
“name”: “Leg_Rear_Left”,
“mesh”: 1
},
{
“name”: “Leg_Rear_Right”,
“mesh”: 1
}
],
“meshes”: [
{
“name”: “Chair_Seat”,
“primitives”: [
{
“attributes”: {
“POSITION”: 0,
“NORMAL”: 1,
“TEXCOORD_0”: 2
},
“indices”: 3,
“material”: 4
}
]
},
{
“name”: “Leg”,
“primitives”: [
{
“attributes”: {
“POSITION”: 5,
“NORMAL”: 6,
“TEXCOORD_0”: 7
},
“indices”: 8,
“material”: 9
}
]
}
],
“accessors”: [
{
“bufferView”: 0,
“byteOffset”: 0,
“componentType”: 5126,
“count”: 8,
“type”: “VEC3”
},
{
“bufferView”: 1,
“byteOffset”: 0,
“componentType”: 5126,
“count”: 14,
“type”: “VEC3”
},
{
“bufferView”: 2,
“byteOffset”: 0,
“componentType”: 5126,
“count”: 20,
“type”: “VEC2”
},
{
“bufferView”: 3,
“byteOffset”: 0,
“componentType”: 5125,
“count”: 36,
“type”: “SCALAR”
},
{
“bufferView”: 4,
“byteOffset”: 0,
“componentType”: 5126,
“count”: 8,
“type”: “VEC3”
},
{
“bufferView”: 5,
“byteOffset”: 0,
“componentType”: 5126,
“count”: 14,
“type”: “VEC3”
},
{
“bufferView”: 6,
“byteOffset”: 0,
“componentType”: 5126,
“count”: 20,
“type”: “VEC2”

@Fennec @Lawrence3DPK This is what the AI generated. A few small adjustments on orbital controls and sphere velocity but it worked pretty well. AI code request - “Create a 3d scene with three.js add orbital controls and 100 random colored spheres animate the spheres in random direction in a area add spotlight add shadows add ambient light spheres should move a medium to slow velocity”

        const scene = new THREE.Scene();

        // Set up the camera
        const camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
        camera.position.z = 5;

        // Set up the renderer
        const renderer = new THREE.WebGLRenderer();
        renderer.setSize(window.innerWidth, window.innerHeight);
        document.body.appendChild(renderer.domElement);

        // Set up the orbital controls
        const controls = new OrbitControls(camera, renderer.domElement);

        // Create and add the spheres to the scene
        const sphereGeometry = new THREE.SphereGeometry(0.5, 32, 32);
        for (let i = 0; i < 100; i++) {
            const sphereMaterial = new THREE.MeshLambertMaterial({ color: Math.random() * 0xffffff });
            const sphere = new THREE.Mesh(sphereGeometry, sphereMaterial);
            sphere.position.x = Math.random() * 10 - 5;
            sphere.position.y = Math.random() * 10 - 5;
            sphere.position.z = Math.random() * 10 - 5;
            sphere.velocity = new THREE.Vector3(Math.random() * 0.2 - 0.1, Math.random() * 0.2 - 0.1, Math.random() * 0.2 - 0.1);
            scene.add(sphere);
        }

        // Create the spotlight with shadows
        const spotLight = new THREE.SpotLight(0xffffff);
        spotLight.position.set(10, 20, 30);
        spotLight.castShadow = true;
        scene.add(spotLight);

        // Enable shadows on the spheres
        scene.traverse(function (node) {
            if (node instanceof THREE.Mesh) {
                node.castShadow = true;
            }
        });

        // Add ambient light
        const ambientLight = new THREE.AmbientLight(0xffffff, 0.5);
        scene.add(ambientLight);

        // Animate the spheres
        function animate() {
            requestAnimationFrame(animate);

            // Move the spheres
            scene.traverse(function (node) {
                if (node instanceof THREE.Mesh) {
                    node.position.add(node.velocity);

                    // Check boundaries and change direction if necessary
                    if (node.position.x < -5 || node.position.x > 5) {
                        node.velocity.x *= -1;
                    }
                    if (node.position.y < -5 || node.position.y > 5) {
                        node.velocity.y *= -1;
                    }
                    if (node.position.z < -5 || node.position.z > 5) {
                        node.velocity.z *= -1;
                    }
                }
            });

            renderer.render(scene, camera);
        }
        animate();
3 Likes

No, it currently has not. I asked about it directly in the conservation with him. “It” can give information about itself and has admitted to me that this possibility can provide even better results in the future.

1 Like

Certainly very impressive! After experimenting with ChatGPT quite a bit over the last several weeks, I’ve found that it’s a wonderful learning tool. It is not sufficiently reliable in itself as yet to be a threat to developers — though it may well be that one day. For now, I like it as a great complement to the documentation, as it usually generates very helpful examples. One thing I have noticed is that it seems to be pretty much useless as an aid to debugging: for any even moderately complex problem it tends to point to complete irrelevancies.

2 Likes