state-spaces/mamba

Mamba is a Python library implementing a new type of neural network architecture designed to handle sequences of data, such as text, audio, or time series, more efficiently than the standard Transformer approach. Transformers, which power most modern AI language models, become slower and more memory-hungry as sequences get longer because every element must attend to every other element. Mamba uses a different mechanism called a selective state space model (SSM), which processes sequences in a way that scales more efficiently with length. The repository provides three generations of the architecture: the original Mamba, Mamba-2 (which introduces a mathematically cleaner formulation connecting state space models and attention), and Mamba-3 (an inference-focused improvement). Each can be used as a building block inside larger neural network models. Pre-trained language models of various sizes are available for download and testing. Using Mamba requires a Linux system with an NVIDIA GPU and a compatible version of PyTorch installed. The library is installable via pip. The project was developed by Albert Gu and Tri Dao, with subsequent work adding the Mamba-2 and Mamba-3 variants. It is intended for researchers and engineers building or experimenting with sequence modeling systems.