blaizzy/mlx-audio

MLX-Audio is a Python library for working with speech on Apple Silicon Macs. It supports three modes: text-to-speech (converting written text into spoken audio), speech-to-text (transcribing spoken audio into written text), and speech-to-speech (transforming one piece of spoken audio into another). The library is built on MLX, Apple's machine learning framework designed specifically for the M-series chips found in modern Macs, so it runs efficiently on that hardware. The text-to-speech side supports a long list of models with different trade-offs between speed, quality, and language coverage. Options include Kokoro for fast multilingual output, OpenAI's Whisper model on the speech-to-text side, and models that support voice cloning, where the system learns the characteristics of a speaker from a short sample and then generates new speech in that voice. Speed of delivery can be adjusted, and the output audio can be streamed in real time rather than waiting for the entire file to generate. For developers, there is a Python API for embedding the functionality in code, a command-line tool for generating audio from a terminal command, and a REST API that is compatible with OpenAI's audio endpoints, making it a drop-in option for projects that already use that interface. A web interface with a 3D audio visualization is also included. The library supports quantization, which is a technique that compresses AI models to use less memory and run faster, with options ranging from 3-bit to 8-bit. There is also a Swift package for integrating the functionality into iOS and macOS apps. Installation is through pip, the standard Python package manager. The library is released under the MIT license and hosted on PyPI.