princeton-nlp/simcse

SimCSE is a Python library from Princeton NLP that turns sentences into numerical representations called embeddings. An embedding is a list of numbers that captures the meaning of a sentence, so that sentences with similar meanings end up with similar numbers. This makes it possible to compare sentences, find near-duplicates, or search a large collection of text by meaning rather than by exact keyword match. The core research contribution is a training method called contrastive learning. In the unsupervised version, the model is shown the same sentence twice and learns to produce consistent embeddings for it, using a technique called dropout as the only source of variation. In the supervised version, the model also uses labeled sentence pairs: pairs labeled as "entailment" (same meaning) are used as positive examples, while pairs labeled as "contradiction" are used as hard negatives to push dissimilar sentences apart. Using the library is straightforward. You install the simcse package, load one of the pre-trained models with two lines of code, and then call methods to encode sentences, compare two groups of sentences by similarity score, or build a searchable index. The repository also supports faiss, a library for fast similarity search over large collections, though with noted compatibility issues on certain newer GPU models. Pre-trained models are available in several sizes and are hosted on HuggingFace. Both unsupervised and supervised variants are provided for BERT-base, BERT-large, RoBERTa-base, and RoBERTa-large backbones. The supervised models score higher on standard sentence similarity benchmarks. The models can be loaded either through the simcse package or directly through the HuggingFace transformers library. The repository also includes training code so researchers can train their own SimCSE models on custom data, along with evaluation scripts and a small demo website that shows sentence retrieval in action.