Wrist Tracking - Ecommerce Try On

Showcase
1

Three.js Real-Time Wrist Tracking AR Using TensorFlow

I’m sharing a Wrist AR Showcase prototype built with Three.js that demonstrates real-time wrist tracking and AR content rendered directly on the user’s wrist using TensorFlow.

watch-gif-1

What It Does

This project tracks a user’s wrist in real time and attaches 3D content (watch models) that follow wrist position and orientation using TensorFlow. It is ideal for wrist-mounted UI, AR wearables, product visualization, and immersive WebAR experiences.

TensorFlow Tracking

Setup

You can set up wrist tracking in your project using TensorFlow with the following scripts:

<!-- TensorFlow Core and Pose Detection -->
<script src="assets/js/tensorflow/tf-core.min.js"></script>
<script src="assets/js/tensorflow/pose/tfjs-converter.js"></script>
<script src="assets/js/tensorflow/pose/tfjs-backend-webgl.js"></script>
<script src="assets/js/tensorflow/pose/pose-detection.js"></script>
<script src="assets/js/tensorflow/pose/pose.js"></script>

<!-- Hand Pose Detection -->
<script src="assets/js/tensorflow/hands/hand-pose-detection.js"></script>
<script src="assets/js/tensorflow/hands/hands.js"></script>

Source Code

The full project, including wrist tracking setup and Three.js integration, is available on GitHub:
:backhand_index_pointing_right: Wrist Tracking Source

Repository Includes

  • Wrist detection and tracking logic

  • Three.js integration

  • Wrist-anchored AR transforms

Demo (Markers & Helpers)

https://theneoverse.web.app/#threeviewer&&watch

Live Wrist AR Demos

You can try multiple wrist-mounted AR watch demos here (mobile recommended):

How It Works (Tech Overview)

  • Wrist keypoint tracking using machine learning

  • Pose data mapped to Three.js transforms

  • AR objects bound to wrist orientation

  • Runs directly in the browser with no app install

3D Orientation Function

// ASSET ORIENTATION
function assetTransform(d) {
    _pm.tracking = _pm.tracking ? _pm.tracking : { coordinates: {}, positions: {}, anchors: {}, object: {} };

    if (d.type == "dot") {
        if (_pm.p.external.dragging) {
            d.p = _pm.math.screenToWorld({ x: d.position.x, y: d.position.y });
            d.m = scene.getObjectByName(_pm.p.models[_pm.p.mIndex.selected].name);

            if (d.p && d.m) {
                if (!_pm.tracking.object[d.id]) {
                    _pm.tracking.object[d.id] = d.mesh =
                        new THREE.Mesh(
                            new THREE.SphereGeometry(0.005, 8, 4),
                            new THREE.MeshBasicMaterial({
                                color: d.id.match(/wrist/) ? 0xff0000 :
                                       d.id.match(/middle/) ? 0x00ff00 : 0x0000ff
                            })
                        );
                    scene.add(d.mesh);
                }

                _pm.tracking.object[d.id].position.copy(d.p);

                if (d.action.match(/dragging/) && _pm.p.external.dragging.anchors.includes(d.id)) {
                    _pm.tracking.origin = _pm.tracking.origin || d.id;
                    d.m.position.lerp(_pm.tracking.object[d.id].position, 0.86);
                }
            }
        }
    }

    if (d.type == "hand" && d.action.match(/tracking/)) {
        d.sm = _pm.p.models[_pm.p.mIndex.selected];
        d.m = scene.getObjectByName(d.sm.name);
        if (!d.m) return;

        d.m.visible = !d.halt;
        register.animation.update = true;
    }
}

Use Cases

1. AR Product Try-Ons

  • Visualize wrist-worn products like watches, bracelets, or smartbands in real time.

  • Allow users to preview size, style, and color before purchasing.

  • Combine with e-commerce platforms for immersive shopping experiences.

2. Wrist-Based HUDs and Notifications

  • Display notifications, health metrics, or navigation info on the user’s wrist.

  • Create wrist-mounted dashboards for smart devices, games, or AR workspaces.

  • Integrate with wearables for seamless interaction without touching the screen.

3. Fitness and Gesture Tracking

  • Track wrist movements for exercise, yoga, or sports applications.

  • Monitor repetitions, form, or gesture-based commands.

  • Use motion data to trigger real-time visual feedback in AR.

4. Interactive AR Interfaces

  • Use wrist gestures to control AR UI elements or menus.

  • Trigger actions like rotating, scaling, or swapping AR objects by hand movement.

  • Create immersive experiences where the wrist acts as a natural controller.

5. Educational and Training Tools

  • Simulate medical, mechanical, or technical procedures with AR overlays on the wrist.

  • Provide step-by-step guidance in real time, anchored to the user’s hand position.

6. Gaming and Entertainment

  • Implement wrist-controlled AR game mechanics.

  • Animate virtual objects on the wrist that respond to player motion.

  • Create collectible or interactive AR wrist items for gamified experiences.

7. WebXR and Cross-Device AR

  • Run directly in a browser with no app installation.

  • Combine with mobile WebXR or AR glasses for multi-device experiences.

  • Enable collaborative AR applications where users see wrist-anchored objects in shared spaces.