How to combine threeJS with AI tools

Questions
,
1

Hi Dear Three Enthusiasts,

I was wondering how to combine threejs with AI tools. For example, if I add a cuboid in the ThreeJS scene, can I combine that with some AI tool (Python based for ex.) To give me automatically the different dimensions of the cuboid (in all dimensions : a, b and c).

This would be interesting instead of manually measure these distances by some distance tool in threeJS.

Let me know your thoughts

Thank you

2
  1. Create a webapp (in Three or anything else, since the rendering part will likely not affect the AI much.)
  2. Create an API in Python, exposing web endpoints that take in some data, run it through your AI pipeline, and return the result back to the webapp.

As to how and what data you’d send to the Python API - depends on the AI models you’d be using :eyes: The GPT-like models have quite poor spatial-awareness, so they may not be very effective, unless you’d transform your scene into vectors and just ask GPT to do calculations on them (I’d still test it thoroughly for longer, continuous conversations - even when given conversation history, GPT often doesn’t see proper relations within 3D spaces, and will not notice it unless you point it out.)

3

Thanks for your input mjurczyk

It seems that the current AI tools are still behind when it comes to 3D spatial recognition and reasoning.

I will make these commands/algorithm manually rather than relying on AI for now…