3D Image Evolution via Partial Differential Equations

Hi,

First of all, thanks to the three.js community for enabling projects like this.

I’ve developed an interactive 3D visualization that demonstrates image evolution under partial differential equations (PDEs), powered by this incredible library.

This web lets you upload your own images and apply various PDEs such as the heat or wave equation, while adjusting boundary conditions, discretization schemes and color maps in real time, among others.

The simulation aims to help in understanding the behavior of these equations from a physical perspective, applied to my field of expertise, image processing.

I’d love to get some feedback on anything; this is a work in progress, and it would help me to improve my undestanding about this framework too.

Check out the demo and source code.

Alejandro.

3 Likes

It’s nice.

I can sense some R/ggplot or Python/matplotlib influence in the color maps.

I’m not sure how hard it would be to readjust the mesh dynamically. Currently the granularity of the mesh geometry introduces edges along curved shapes. Maybe this also causes some strange patterns, like this:

However, this could by an issue only for synthetic images with curved lines. For photos of Nature this might not be an issue.

Do you plan to move the calculations to the GPU? This is not trivial, and would require some efforts, but it would boost performance significantly.

1 Like

Hi Pavel, thanks for the insights.

You’re right, I’ve noticed those visual artifacts in the wireframe connectivity, especially when the image flattens or values approach zero. It’s an effect I’m actively looking to mitigate. I suspect it’s related to how I’m downscaling the original image resolution to reduce the domain size. Possibly switching to bilinear interpolation or a similar method could help smooth out those patterns.

Three.js might also offer some post-processing tools like antialiasing or smoothing filters that could improve the visual quality of the material and standard mesh.

I hadn’t seriously considered moving the numerical computation to the GPU, and it’s a great idea. It could definitely speed up the current JavaScript CPU implementation, which is the main bottleneck affecting the simulation’s FPS. Running solver steps on a shader it’s still an unknown for me, but definitely interesting.

Instead, I did think about launching a Web Worker to run the solver separately, but I haven’t had much success yet, mostly due to my limited familiarity with some of the ES module system quirks. Still, I’m working on it; if you have any related resources or tips, I’d really appreciate it.

1 Like

I like doing these too. I have one that’s pretty similar here. Also one of my first Three.js posts.

I also have a GPU version here if you want to check it out.

Yours is nice, hope to see more!

2 Likes

Hi Chris,

I’ve definitely been inspired by your FDM simulation, amazing work! I absolutely need to take a closer look at your GPU implementation. Thank you for sharing it, those FPS numbers look very promising. As soon as I make progress with the GPU implementation, I’ll be sure to share my results.

1 Like

Please share any updates! Your version is a quite an improvement over mine. The GPU PDE is written in shader code, once you get the hang of that, it’s mostly working with Three.js buffers.

Here’s a shader GPU example I wrote a while ago. (I was learning as I wrote it, so take it with a grain of salt)

1 Like

As an update: I’ve been testing GPU acceleration in Three.js with a much more simplified heat equation simulation. Check:

https://codesandbox.io/p/sandbox/heat-equation-gpu-cpu-7r9w8n

At 2048x2048 resolution, results are:

  • GPU (WebGL2 shaders): ~110 FPS

  • CPU (JS processing): ~30 FPS.

I’ve used ping-pong buffers for GPU version, which minimizes the CPU/GPU data transfer bottleneck. I think that is the major drawback in my original simulation.

Would love to hear different workarounds to tackle it :monkey_face:.