I don’t think this can be done with GLSL. You’ll need an AI-based background remover (and a watermark remover for the last one), combined with a 2D canvas context.
A quick search for “JS AI background remover” gave me these links:
thanks for the quick response, I have just shared a set of images downloaded from freepik, watermark removal is not a part of requirement.
I need to convert many images with a transparent surface, so I am looking for an open-source option. unfortunately imgly third option seems to be a paid one.
could you please share which approach you have used to convert those sample images ?
And if you have access to a GPU - setting it up is even easier, you just install ComfyUI, add a single node (see example workflow at the bottom), and expose it all as an API endpoint.
I’ve used Imgly ! But as far as I know, it’s not under a paid license, they use AGPL-3.0, which is open source with a strict copyleft license. That might not align with what you’re looking for.
The model is open-source but only for non-commercial use.
Commercial use requires a separate agreement with BRIA. Contact Us for more details.
As for the second link, you can ignore it. Now that I’ve taken a closer look, it’s based on BodyPix, so it only supports body segmentation.
After some digging, I found this blog post on image segmentation. It uses Google’s DeepLab, licensed under Apache-2.0, so it could be a good alternative.
Effective, adequate and universal removal of background in complex images is not trivial. I think it is unsolved problem … there are lot of tools to remove background, but each of them will fail in some cases. Such removal would require either understanding of the objects and their spacial properties, or would require some help (like different colors, depth perception, image recognition, human intervention, etc). That’s why movie productions still use green screens. It just makes things much more manageable.
Here is a sketch that might give you a perspective of potentially fast way to deal with the issue:
And a warning: if you plan to harvest images of furniture or other objects from online images, you should carefully check the license. My suggestion is to contact the other party before initiating mass harvesting even if the images are declared as “free”.
main usage of approach will be user will upload the actual real time images of their own house / room, from that uploaded images we need to clear the surface backgrounds. Shared images are for the sample usage and testing purpose only.
yes, Agreed.
As I am trying for the dynamically uploaded images, removing bg with manual efforts won’t work. I have already spent a lot of time searching for a suitable library to handle this, but haven’t found one. Writing my own code would take too long, especially since I am still in the learning phase.
and… extending this answer, the Depth-Anything model can be run in the browser…
from depth, you could reconstruct normal, and from normal, extract planes…
from user selected planes, you could extract a mask with the rest of the depth buffer.
using a segmentation model directly is probably easier to implement though.
! But as far as I know, it’s not under a paid license, they use 