fluxions-ai/stftvae

STFT-VAE is a Python library for compressing and reconstructing audio using a neural network approach. Its job is to take a sound recording, squeeze it down into a much smaller, compact representation, and then rebuild the original sound from that compact version, a process often called audio coding. What makes this project different is how it handles the compression step. Many audio codecs break sound into a fixed set of discrete codes, similar to picking from a limited dictionary of sound chunks. STFT-VAE instead keeps the compressed representation continuous, more like a smooth set of numbers than a fixed vocabulary, and it relies on a well established audio transform, the short time Fourier transform, to preserve the timing and phase information of the sound directly, rather than asking a separate model to guess it back afterward. The pretrained model included with the project takes audio recorded at 24,000 samples per second and compresses it down to roughly three updates per second, a very low rate compared to similar tools, while still reconstructing speech that the project's own comparisons show sounds as good as or better than some well known alternatives. Using the library is straightforward for someone comfortable with Python. You install it with pip, load a pretrained version of the model from Hugging Face, and either run a full round trip that compresses and rebuilds an audio clip in one step, or run the compression and reconstruction steps separately if you need direct access to the compact representation. There is also a command line tool for reconstructing audio files directly, and a small web based demo built with Gradio that lets anyone upload or record audio and listen to how it sounds after passing through the codec, without writing any code. The project includes scripts to reproduce its own quality comparisons against other audio codecs, and it is released under the MIT license, meaning it can be freely used, modified, and shared, including for commercial purposes.