How AI Background Removal Actually 'Sees' a Photo

The old way: tracing edges

Before machine learning became practical for this, background removal tools worked by detecting hard edges and color boundaries — essentially guessing where a subject ends and the background begins based on contrast. This worked reasonably well for a product photographed on a plain white background, but broke down badly on a busy or low-contrast background, or anywhere the subject's edge wasn't a clean, high-contrast line.

The modern way: a per-pixel salience prediction

A modern AI segmentation model takes a completely different approach: instead of tracing edges, it was trained on a large number of example photos where humans had already marked exactly which pixels belonged to the main subject, and it learned to recognize what generally makes something "the subject" of a photo — regardless of whether that subject is a person, a product, an animal, or an everyday object, and regardless of what's actually behind it.

Confidence, not a hard yes/no

The model's actual output for each pixel isn't a simple "subject" or "not subject" label — it's a confidence score between 0 and 1, representing how certain the model is. A pixel deep inside the main subject might score 0.99, while a wispy strand of hair or fur at the very edge might score 0.4, reflecting genuine visual ambiguity. Using that confidence value directly to control transparency (rather than forcing every pixel to a hard in-or-out decision) is what produces softer, more natural-looking edges around hair and other fine detail — though a general-purpose model like this can still be thrown off by a cluttered background or a low-contrast subject.

Why it still runs entirely in your browser

Running this kind of model traditionally required a server with real computing power, meaning photos had to be uploaded somewhere for processing. Modern browser technology (WebAssembly) now lets a pre-trained model run directly on your own device at reasonable speed, which is what lets a tool like this remove a photo's background without the image ever leaving your computer.

Try it yourself

Our Remove Background tool uses exactly this approach — IS-Net, an open-source general-purpose dichotomous image segmentation model, running entirely on your device via ONNX Runtime.