zalo/whisperlive

WhisperLive is a tool that turns spoken audio into written text almost instantly. It lets you transcribe live speech from a microphone or pre-recorded audio files, and it can also translate spoken languages into English. The project provides a server that processes audio and a client that sends audio to the server, with browser extensions and an iOS app available for easy access. At a high level, it works by running OpenAI's Whisper speech recognition model on a server. The client connects to that server and streams audio to it, either from a microphone, an audio file, or even a live video stream. The server supports multiple backends for processing the audio, including faster-whisper, NVIDIA's TensorRT, Intel's OpenVINO, and Mistral AI's Voxtral, giving you flexibility depending on your hardware. The client can specify the language, whether to translate, and which model size to use. Anyone who needs real-time captions or transcription would find this useful. A product manager building an accessibility feature for a video platform could use it to generate live subtitles. A founder creating a meeting-notes tool could route audio through WhisperLive to capture what was said. A journalist could use it to transcribe interviews on the fly. The browser extensions make it practical for everyday users who want captions while watching videos online. The project is built with flexibility and performance in mind. The Docker setup simplifies deployment across different hardware configurations, including GPUs and Intel-based systems. It includes voice activity detection to improve transcription accuracy by filtering out silence. The server can be configured to handle a specific number of simultaneous clients and limit connection times. The README notes that translation to languages beyond English is planned for future development.