I’ve been making short animated comedy episodes (1080x1920, for Reels/Shorts) and built the renderer on three.js. Sharing how it works, since a few parts might be useful to others.
The idea: an episode is a JSON script, not a timeline. Each line says who speaks, the emotion, an optional gesture and the shot:
{ "speaker": "rudy", "emotion": "happy", "text": "Do you have abs? To go?", "action": "lean_in", "shot": "two" }
How it renders:
- A three.js page runs in headless Chrome and draws the episode frame by frame. Every frame is computed from time, not from wall-clock playback, so a slow frame never drops and the same script gives the same frames.
- The voices are generated first; their durations drive the timing of the lines, the shots and the mouth animation.
- Characters and sets are pre-built GLB models (CC0 packs from Quaternius, Kenney and KayKit, plus my own). Emotions and gestures are spec files, and a rule-based planner picks close / two-shot / over-the-shoulder framing.
- Captions are drawn word by word, then ffmpeg muxes frames, voices, music and SFX.
- A QC pass checks loudness (-14 LUFS), black or frozen frames and each app’s safe areas, so captions and faces stay clear of the UI buttons.
About a minute of render time per 20 seconds of video on a mid-range laptop GPU.
Because the cast is real 3D models, the characters look the same in every episode, which was the whole reason I didn’t use a video model.
You can poke at the asset browser and the episode player in your browser (no install, runs on three.js):
Happy to go into any part: the frame clock, mouth animation, the shot planner or the safe-area checks. (I’m packaging it as a kit for coding agents, but the questions here are about the three.js side.)
