openmoss/moss-transcribe-diarize

MOSS-Transcribe-Diarize is an open source AI model that turns long audio recordings, like meetings, phone calls, podcasts, interviews, and lectures, into a written transcript that also identifies who is speaking and when. Instead of running separate tools for converting speech to text and for figuring out which speaker said what, this single model does both jobs at once, producing text with precise timestamps and consistent labels for each speaker, such as speaker one and speaker two. The model supports more than fifty languages and is designed to handle long, messy recordings with multiple people talking, which is where many transcription tools struggle. It can also optionally flag acoustic events, giving extra context about what is happening in the audio beyond just the words spoken. The output format is compact: each segment shows a start time, a speaker tag, the transcribed text, and an end time, all strung together in one continuous stream. Under the hood, the model combines a text processing component based on a smaller Qwen language model with an audio understanding component based on Whisper's audio encoder, joined together so that audio information can be merged directly into the text generation process. There is also a stronger paid version called Pro that will be available through an API. The project shows benchmark comparisons against other transcription systems, including commercial ones, across several test sets covering different types of audio like meetings and movies, and the smaller open source model performs competitively while the Pro version leads on most measures. The team reports the model recently won first place in a multilingual speech recognition challenge at a major speech conference. To try it, you set up a clean Python environment using a tool called uv, install the required packages, and then run the provided Python code. The model can also be served for higher throughput using either the SGLang or vLLM serving frameworks, and there is a web based subtitle app for a more visual way to use it.