nvlabs/segformer

SegFormer is an AI tool from NVIDIA that performs "semantic segmentation", meaning it looks at an image and identifies exactly which pixels belong to which object or scene type. For example, it can look at a street photo and label every pixel as road, person, car, sidewalk, building, or sky. It comes in several sizes (B0 through B5), letting you trade off between speed and accuracy depending on your needs. Under the hood, it uses a "transformer" architecture, the same family of AI models behind modern language tools, but adapted here for visual understanding instead of text. The project is built on top of a popular open-source codebase called MMSegmentation. The repository provides pre-trained models (weights you can download and use directly) along with scripts to train new models on your own image datasets or evaluate how well existing models perform on standard benchmark datasets like ADE20K and CityScapes. This tool is aimed at developers and researchers working on computer vision applications, particularly autonomous driving, where a car needs to understand what's around it, or scene understanding for robotics and augmented reality. A startup building navigation for delivery robots, for instance, could use it to help the robot distinguish between a walkable path and an obstacle. It was published as a research paper at NeurIPS 2021, so it's primarily designed for research and evaluation rather than production deployment. One important caveat: the license is non-commercial only. You can use it freely for research or evaluation, but if you want to build a product around it, you'd need to contact NVIDIA for a commercial license.