koljab/realtimevoicechat

RealtimeVoiceChat lets you have a spoken conversation with an AI language model through your web browser. You speak, it listens and responds in near real-time with synthesized speech, and you can interrupt it mid-sentence just as you would in a normal conversation. The flow works like this: your browser captures your microphone audio and streams it over a WebSocket connection to a Python backend. The backend transcribes your speech to text using a library called RealtimeSTT, sends the text to an AI language model for a response, then converts the response back to speech using RealtimeTTS and streams the audio back to your browser. The whole pipeline is designed to keep the delay between you finishing a sentence and the AI starting its reply as short as possible. The AI language model backend is pluggable. By default it connects to Ollama, a tool for running open-source AI models locally on your own machine. You can also configure it to use OpenAI's API instead. For speech synthesis, you can choose between several text-to-speech engines: Kokoro, Coqui, or Orpheus. The turn-detection logic watches for pauses in your speech to decide when you have finished talking, and it adapts to the pace of the conversation. Running this project requires a reasonably powerful NVIDIA graphics card (GPU). Without one, the speech recognition and synthesis models run much more slowly and the real-time feel breaks down. The recommended setup uses Docker on Linux, which bundles the application and its dependencies into containers. A Windows installation script is also provided. The backend is built with Python and FastAPI. The original author is no longer actively adding features or providing support, and the project is now community-maintained. Pull requests from contributors are still reviewed and merged periodically.