Audio-Reactive 3D Visualizer — Three.js + Web Audio API with In-Browser MP4 Export

Showcase
, ,
1

Hi everyone!

I’m Nagisa Dozono, a frontend developer and music producer from Japan.

I’ve built an open-source, browser-based audio-reactive 3D visualizer using Three.js, WebGL, and the Web Audio API.

The project started from a problem I faced as an independent musician: producing a complete music video can be expensive, and it can be difficult to communicate the exact timing, atmosphere, and visual direction of a song to another creator.

This tool lets users upload their own music and artwork, generate synchronized visuals in real time, and export the result as a 1920×1080 MP4 directly in the browser.

Main features

  • Real-time volume, frequency, and waveform analysis using the Web Audio API

  • Audio-reactive 3D rendering with Three.js and WebGL

  • Multiple 3D, particle, waveform, and image FX modes

  • Adjustable particle count, size, shape, camera distance, and morph intensity

  • Customizable visual layer order

  • Fullscreen Live / VJ mode with keyboard controls

  • Browser-based MP4 export using WebCodecs, with ffmpeg.wasm as a fallback

  • Audio, image, and video processing performed locally in the browser

Tech stack

  • React

  • TypeScript

  • Vite

  • Three.js / WebGL

  • Web Audio API

  • WebCodecs

  • ffmpeg.wasm

  • Zustand

  • Material UI / Emotion

  • Cloudflare Workers Static Assets

Links

Live demo:

GitHub repository:

Demo video / post:
https://x.com/nagisa7g/status/2066262368234225844

The source code is available under the MIT License.

I’d especially appreciate feedback on the Three.js implementation, rendering performance, audio-reactive behavior, visual design, and overall UX.

Contributions, issues, stars, and pull requests are also very welcome.

Thanks for taking a look!

2 
nice work

CodePen Embed - Random Dancing Girl

CodePen Embed - Audio Player with Real-Time Shader Visualizer

I also tried something like that, but it’s always better for the musician to create the animation after the music is finished, instead of at runtime.

3

Thanks for sharing these — the shader visualizer is really nice.

I agree that, for an actual music release, the final animation should usually be created from the finished track rather than depending only on live runtime behavior.

The real-time mode in my project is mainly used as a preview and editing environment. The musician can load the completed track, adjust the scene, particles, effects, sensitivity, and layer order while listening, and then export a final 1920×1080 MP4 directly in the browser.

So the workflow is essentially:

finished track → real-time visual adjustment → deterministic final video export

The Live / VJ mode is an additional use case rather than the only intended workflow.

Thanks for taking a look and for sharing your experiments!

4

That makes it much better. It’s practically an editor.

5

Exactly — that’s the direction I was aiming for.

It works as a real-time visual editor first, then lets the musician export the finished result as a video. I’m planning to keep improving the editing workflow and add more reusable presets over time.

Thanks for checking it out!

6

Small follow-up: I’ve also released a minimal educational version of the core audio-reactive pipeline.

Web Audio Three.js Starter focuses on the smallest understandable setup for:

  • Loading a local audio file

  • Analyzing volume, bass, mids, highs, and waveform data

  • Mapping those signals to a Three.js mesh

  • Driving a particle system in real time

  • Keeping high-frequency audio updates outside normal React rerenders

Repository:

Live demo:

The full visualizer is still here:

I made the starter for developers who want to study or reuse the core Web Audio API + Three.js architecture without the larger editor and export features.