Best production-ready face tracking solution for Three.js WebAR on iOS & Android browsers?

Questions
, , , , , ,
1

Hi everyone,

I’m currently researching the best approach for implementing real-time face tracking in a commercial Three.js/WebGL web application, and I’d really appreciate feedback from developers who have built and shipped similar systems.

The important requirement is that this must work directly in the browser, without requiring a native mobile app.

ChatGPT Image Sep 5, 2026, 06_35_17 PM
ChatGPT Image Sep 5, 2026, 06_35_17 PM1659×948 354 KB

Requirements

The experience should work reliably on:

  • iOS Safari
  • Android Chrome
  • Modern mobile browsers in general

The main use case is similar to Instagram/Snapchat-style AR face effects, where we need to track a user’s face in real time and use the tracking data to drive Three.js content.

For example:

  • Face landmarks / keypoints
  • Head position and rotation
  • Stable face tracking while the user moves
  • Facial expressions / blendshapes if available
  • Attaching 3D objects such as glasses, masks, hats, etc.
  • Face filters and visual effects
  • Potentially face/skin effects or segmentation

What I’m evaluating

I’m currently looking at solutions such as:

  • MediaPipe
  • TensorFlow.js
  • Jeeliz
  • Banuba
  • 8th Wall
  • WebXR
  • Other WebAssembly/WebGPU-based solutions

I’m particularly interested in real-world production experience, rather than just demos or benchmarks.

For anyone who has implemented WebAR face tracking with Three.js:

What solution would you choose today for a commercial project, and why?

I’d especially appreciate feedback regarding:

  • iOS Safari reliability
  • Android browser performance
  • Tracking stability and jitter
  • CPU/GPU and battery usage
  • Latency
  • Integration with Three.js/WebGL
  • Browser/device compatibility
  • Licensing and commercial costs
  • Ease of maintaining the solution long-term

Also, are there any solutions you tried that looked good in a demo but caused problems in production?

If you were starting a new Three.js + WebAR face-effects project in 2026, what stack would you choose?

Thanks in advance for sharing your experience!

2

I’ve had good results with blazepose:

Also this:

3

For a commercial product the deciding number is not accuracy — all of them track well enough for glasses. It is what loads before the first frame. Measured today:

  • @mediapipe/tasks-vision@1.0.1 bundle — 155 439 B
  • vision_wasm_internal.wasm (SIMD) — 11 756 954 B
  • face_landmarker.task (float16) — 3 758 596 B

About 15.5 MB before it tracks anything, none of it cached from elsewhere. On mobile data that is your first impression.

The thing that makes MediaPipe worth it anyway: it returns facialTransformationMatrixes — a ready 4×4 for head pose. You don’t fit it from landmarks yourself:

new THREE.Matrix4().fromArray(m.data).decompose(glasses.position, glasses.quaternion, glasses.scale);

It’s off by default — outputFacialTransformationMatrixes: true, plus outputFaceBlendshapes: true for the 52 ARKit coefficients.

TF.js face-landmarks-detection isn’t a separate option: with runtime: 'mediapipe' it’s the same graph behind a wrapper. Jeeliz is the honest pick when 15 MB is unacceptable — much smaller, but no blendshapes and no head matrix. Banuba and 8th Wall solve iOS for you; decide those on the licence, not the tech.

One iOS detail that costs an afternoon and looks like a library bug: the video element needs playsinline and muted, or Safari hijacks it into the fullscreen player and your canvas samples nothing.

Happy to put the MediaPipe path on a page so you can open it on the actual iPhone you need to support instead of trusting a table.