idea-research/grounded-segment-anything

Grounded-Segment-Anything is a project from IDEA Research that wires several open-source AI vision models together into one pipeline. The idea is to point at any object in an image just by typing a word, for example, "the red bag", and have the system find that object, draw a tight outline around it, and optionally edit or describe it. It does this by chaining specialist models. Grounding DINO is an open-vocabulary object detector, meaning you give it a text prompt and it locates whatever you described in the image with a box, without needing to be retrained for each new category. Segment Anything (SAM) takes those boxes and produces a precise pixel-level mask, the actual outline of the object. The pipeline can then hand that mask to Stable Diffusion to edit the region, or to Recognize Anything (RAM) to automatically generate descriptive tags. The README is explicit that all parts are independent: any piece can be used on its own or replaced with a similar model, like swapping in a different detector or a different image generator. You would reach for it when you need to find and outline objects in images from a text description rather than from labeled training data. The README highlights uses like automatic data labeling, open-vocabulary detection and segmentation, image editing, and data generation. There is also a follow-on project, Grounded SAM 2, for tracking objects across video. The repository is primarily Jupyter Notebooks, so you can run and inspect each stage interactively, and the project lists hosted demos on Hugging Face Spaces, Colab, Replicate, and ModelScope. The full README is longer than what was provided.