yashkotha/whisper-valet

Whisper Valet is a macOS tool that sits alongside MacWhisper, a popular transcription app. You drop a rough audio file into a watched folder, and the valet gives you back cleaned audio plus a transcript that labels who speaks when. Everything runs locally on your machine, so your recordings never get uploaded anywhere. It works on monologues, two person interviews, or multi speaker panels. The pipeline has five stages. First, ffmpeg extracts a uniform audio stream, handling video files too. Demucs, a tool from Meta AI, separates human voices from background noise onto a clean track. The quiet speaker is then amplified to match the loud one. MacWhisper transcribes the cleaned audio with word level timestamps. Finally, pyannote figures out who speaks when using the original audio, and those labels get merged onto the transcript. The split matters: noise removal helps transcription but muddies the vocal characteristics that speaker identification relies on, so the pipeline transcribes the clean version but attributes speakers from the original. Every transcript segment carries a confidence score. Anything below 0.6 gets flagged so you know which lines to double check. If pyannote cannot run because you lack a HuggingFace token or network access, the valet falls back to MacWhisper's built in speaker detection and notes this in a report file. You get several outputs per clip: the untouched original, a cleaned version, an aggressive denoise variant, a labeled transcript, subtitle files, a JSON file with word level timestamps, and a confidence report. The project also includes an MCP server, which lets AI assistants like Claude queue files, check progress, and read transcripts. Queue operations return instantly, so long jobs do not time out. A configuration file lets you set speaker counts, optional names, and notification preferences. The daemon survives reboots and processes files sequentially. The full README is longer than what was shown.