jianchang512/pyvideotrans

pyVideoTrans is an open-source tool that automatically translates videos from one language to another, replacing the original speech with dubbed audio in a new language and generating translated subtitles, all in one workflow. The process works in four steps: first, it listens to the video's speech and converts it to text (ASR, or Automatic Speech Recognition), next, it translates that text into the target language using an AI language model, then it generates new spoken audio from the translated text (TTS, or Text-to-Speech), and finally it combines everything back into a finished video. You can pause and manually correct any step along the way before moving on. The tool supports a wide range of speech recognition engines, including local offline models (Faster-Whisper) and cloud services. For translation, it connects to AI models like DeepSeek, ChatGPT, Claude, Gemini, and Ollama (for fully local, offline translation). For voice generation, it supports options including Microsoft's Edge-TTS (free) and voice cloning models like F5-TTS, CosyVoice, and GPT-SoVITS, which can clone a specific person's voice style. Additional features include speaker diarization (identifying who is speaking when), multi-role dubbing (different AI voices for different speakers), vocal separation, and a command-line interface for batch processing on servers. Windows users can download a ready-to-run executable with no setup. Developers on any platform can run it from source using Python. GPU acceleration via CUDA is optional but speeds up local AI models significantly.