facebookresearch/fairseq

Fairseq is a research toolkit from Facebook AI Research (Meta) for training neural network models that work with sequences, most commonly text. The core problem it addresses is that building and experimenting with state-of-the-art sequence modeling architectures from scratch is enormously time-consuming. Fairseq provides a well-engineered framework with reference implementations of dozens of published research papers, so researchers can reproduce existing results, extend them, or swap in new ideas without rewriting all the surrounding infrastructure. The toolkit covers a wide range of tasks: machine translation (converting text from one language to another), text summarization, language modeling (predicting what word comes next in a sentence), and speech recognition. It also includes multimodal models that work with both video and text. Each task type is paired with multiple model architectures, convolutional neural networks (which process sequences using sliding windows of context), LSTMs (Long Short-Term Memory networks, an older recurrent architecture suited for sequential data), and Transformer models (the attention-based architecture that underpins most modern language AI, including systems like GPT and BERT). Fairseq is designed for researchers who want to train large models efficiently. It supports training across multiple GPUs and machines, gradient accumulation (a technique for simulating larger batches on limited hardware), and memory optimizations like parameter sharding (splitting model weights across devices). The Hydra configuration framework is used to manage the many experiment parameters cleanly. You would use Fairseq when conducting natural language processing research, reproducing results from academic papers, or training custom translation, summarization, or speech recognition models. It requires Python and is built on top of PyTorch, Facebook's open-source deep learning framework. It is not typically used to build end-user products directly, it sits at the research and experimentation layer.