Multiplaying with avatars as stripped background camera streams

Hey, we have made this multiplaying 3D environment where avatars are represented by their web camera streams where the background is removed. It is meant to be used for realy cheap virtual productions. Only a tablet or laptop or mobile phone. We have used NetworkedAframe and Mediapipe Selfie Segmentation @ javascript API.

Play live demo ( two or more players needed):


Networked-Aframe = (Aframe + EasyOpenRTC) is a library for multiplaying in VR

Mediapipe Selfie-Segmentation = GoogleAI at WebAssembly for removing the background of the webcamera stream.

Easy-OpenRTC = an abstraction of WebRTC for p2p connections. Very handy in opposite of Janus adapter.

WebRTC couldn’t transfer transparent pixels (they are automatically become black) so I placed green pixels to be transferred (black can not be easily removed).

I have made my custom shader at client side that replaces highly green pixels to the transparent ones (which means that if you wear a green blouse, then only your head will be visible).

Requirements: At least two players (it is p2p based on WebRTC)
Browser: Firefox or Chrome (in Firefox is more stable)
Device: A GPU is recommended but it runs also without it.
Permissions: It requires Mic and Camera permissions (if you delay to give permissions, it will start without them and your avatar will be a black panel, remedy: open a new window and close the old one, do not press restart)

Dimitrios Ververidis @ CERTH , 8/10/2021

EU H2020 Horizon funding programme


Latest progress: We have a stage now !

1 Like

Hey Jim. Amazing progress!

What solution do you have in mind for many-to-many interactions? See this example: Video meetings in 3D | Shapespark

(with OS solutions/frameworks off-course)

Greetings from Colombia


I target to Virtual Productions from distance enabling actors to play anywhere from the world in a 3D scene.

See the last comment in my pull request for Networked-Aframe main branch: