mudler/locate-anything.cpp

This project is a C++ implementation of NVIDIA's LocateAnything-3B, an AI model that finds objects in images based on text descriptions you give it. You provide an image and a text prompt (for example, "person" or "car"), and the model draws bounding boxes around all matching objects it finds. The result is both a JSON list of detected items with coordinates and, optionally, an annotated image with boxes drawn on it. The model combines a 3-billion-parameter language model with a vision component. It works by converting image coordinates into special tokens, so the AI reads locations the same way it reads words. The team behind this project also maintains LocalAI, a broader open-source tool for running AI models locally without requiring a cloud service. The main advantage over the official NVIDIA release is speed and reduced dependencies. At inference time, no Python installation is required. On a modern desktop CPU, this version runs roughly 1.5 to 3 times faster than the official PyTorch-based model, depending on configuration. On an NVIDIA GPU it runs about twice as fast. The detections produced are identical or nearly identical to the official model, verified by the developers through benchmark comparisons across multiple scenes. To use it, you build from source using CMake, download a pre-built model file from Hugging Face (the recommended q8_0 variant is about 6.3 GB), and run the command-line tool with an image and a text prompt. Multiple categories can be specified in a single prompt by separating them with a special delimiter. There are also a C++ library and a flat C interface for embedding the detector in other software. The project is MIT-licensed. It offers several model file variants of different sizes and precision levels, trading some detection accuracy for smaller file size and faster inference.