Analysis updated 2026-05-18
Try or demo a live speech-to-speech translation pipeline with speaker-aware audio output.
Swap in different ASR, translation, and TTS backends to compare speech translation quality.
Study or extend the reference implementation behind the X-Translator research paper.
| zhaoyx239/x-translator | alexrosbach/replibook | arlandaren/proagents | |
|---|---|---|---|
| Stars | 25 | 25 | 25 |
| Language | Python | Python | Python |
| Setup difficulty | hard | easy | easy |
| Complexity | 4/5 | 2/5 | 1/5 |
| Audience | researcher | ops devops | vibe coder |
Figures from each repo's GitHub metadata at analysis time.
Requires running separate ASR, translation, and TTS backend services before the demo works, and a GPU for the CUDA build.
X-Translator is a research system that translates speech in real time while someone is talking, and speaks the translation back in the target language. It connects three pieces: a speech recognition model that listens to the speaker, a translation model that converts the recognized text, and a text to speech model that reads the translation aloud, all coordinated by a runtime controller. A browser page can show the original text, the translated text, and play the synthesized voice as the conversation happens. The project comes from a research group and includes a published paper describing the design, along with a public online demo you can try without installing anything. This release includes the local demo and the backend adapters that connect to different speech recognition, translation, and text to speech services. The code for formally evaluating the system's accuracy is planned for a later release. The repository is organized into a FastAPI backend that runs the session logic and runtime controller, a simple browser frontend, and a server folder holding adapters and setup notes for each supported speech and translation service. To run it yourself, you set up a Python virtual environment, install the project, and optionally install a CUDA build of PyTorch if you have a compatible GPU. Configuration happens through a single config.json file, where you choose which speech recognition engine to use, such as Qwen3, SenseVoice, Paraformer, or Zipformer, which translation engine, such as LMT or Hunyuan, and which text to speech engine, such as X-Voice or IndexTTS, along with the source and target languages. Those backend services need to be started separately before launching the browser demo with the included start script. The project credits several existing open source speech recognition, translation, and text to speech projects that its adapters connect to. X-Translator's own code is released under the MIT License, though the third-party models and services it can connect to keep their own separate licenses.
A real-time speech-to-speech translation system that listens, translates, and speaks back the translation live, built from a research paper.
Mainly Python. The stack also includes Python, FastAPI, PyTorch.
Use freely for any purpose, including commercial use, as long as you keep the copyright notice, third-party components keep their own licenses.
Setup difficulty is rated hard, with roughly 1h+ to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
Verify against the repo before relying on details.