m-bain/whisperx

WhisperX is an automatic speech recognition tool, software that converts spoken audio into written text. It builds on Whisper, an open speech-recognition model developed by OpenAI, and adds three things Whisper alone does not do well: it runs much faster, it provides accurate timestamps for every individual word rather than for whole sentences, and it can tell different speakers apart and label who said what. The README explains the pipeline. Audio first passes through voice activity detection (VAD), which finds the segments where someone is actually speaking; this both reduces hallucinated transcriptions and allows the audio to be processed in batches. Those batches are then transcribed by Whisper itself (running on a faster backend called faster-whisper) for high throughput, the README cites roughly seventy times real-time speed using the large-v2 model. The resulting words are then aligned to the audio using a phoneme-level model (wav2vec2.0) to produce word-level timestamps that are much more precise than Whisper's native sentence-level ones. Finally, optional speaker diarization (powered by pyannote-audio) partitions the audio by speaker so each line in the transcript can be labelled with a speaker ID. You would use WhisperX whenever you need a high-quality transcript with reliable word timings, for example to generate subtitle files where the text needs to land on the right frame, to build a searchable index of meetings or lectures, or to feed downstream tools that act on individual words. WhisperX is written in Python, distributed on PyPI, and can run on the GPU (with CUDA) or fall back to CPU. Enabling speaker diarization requires a Hugging Face token and accepting the model's user agreement. The full README is longer than what was provided.