I’m currently working on a project with a module dedicated to a virtual try-on feature for various clothing items (shirts, pants, suits, trousers) using augmented reality. This is a web-based application.
I want to utilize technologies such as MediaPipe, Canvas, and Three.js to render and display 3D models of the clothing items over the user’s body. However, I’m facing difficulties with the following aspects:
- *** Body Tracking and Clothing Alignment.**
- *** 3D Model Display.**
If anyone has experience with similar applications or technologies, I would greatly appreciate your guidance!
But that’s 100% of the aspects involved in that kind of application 
While this may not be a direct answer for you, from experience I can suggest you look towards AI and inpainting (combined with segmentation controlnets that’ll allow you to detect clothes on the webcam feed) rather than MediaPipe and tracking. You’ll likely get significantly better results using a <1sec LCM near-realtime generations than you’ll get trying to fit a mesh onto a webcam feed accurately.
1 Like
I wrote something like this before, and it was Hard. Had to do all kinds of crazy stuff. Filtering the stream data… building an IK rig with bones that track the blazepose data. There’s a reason why all the good solutions are commercial.
1 Like