facebookresearch/faiss

Faiss is a library for finding similar items in large collections of vector data very quickly. The problem it addresses is a common need in AI and machine learning applications: given a collection of millions or billions of items represented as numerical vectors (think image embeddings, text embeddings, product representations), find the ones that are most similar to a query item. A naive search would compare the query against every item one by one, which becomes impossibly slow at large scale. Faiss provides algorithms that find the nearest neighbors far faster by using smart indexing structures that allow most of the collection to be skipped. The library contains many different search algorithms, each making different trade-offs between search speed, result accuracy, memory usage, and how long it takes to build the index. For exact results at small scale you can use a flat index that compares everything directly. For billion-scale collections you can use compressed representations that sacrifice some accuracy in exchange for fitting in memory and searching faster. Faiss also includes GPU implementations of many of these algorithms, which can be dramatically faster than CPU-only search. It is written in C++ with full Python and NumPy wrappers so you can use it from either language. You would use Faiss if you are building a semantic search system, a recommendation engine, an image similarity search, a retrieval-augmented generation pipeline where you need to find relevant documents by embedding similarity, or any other application where you need to find the closest vectors in a large dataset quickly. It is developed by Meta's Fundamental AI Research group and is available under the MIT license.