avaturn-live/avtr-1

AVTR-1 is a project that lets you turn a single portrait photo into a talking, reacting avatar that can hold live conversations. You give it a picture of a person and an audio clip, and it produces a video where that person appears to speak and listen in real time, with lip movements that match the audio. The system can handle two-speaker dialogues. Feed it audio from both sides of a conversation and it will generate video of each speaker reacting appropriately, with the avatar looking engaged while the other person is talking. This is built specifically for live use, running fast enough to keep up with a real-time audio stream on modern graphics hardware. Setup requires a Linux computer with an NVIDIA graphics card from the Ampere generation or newer (RTX 3070 and above are listed as supported). You also need CUDA and TensorRT, which are NVIDIA software frameworks for running AI models on graphics cards. The installation process downloads pre-trained model weights from HuggingFace, a public AI model hosting site, then compiles them into fast inference engines specific to your hardware. This compilation step happens once and can take a while. Once installed, you can run the demo interactively or generate video files offline. The offline mode supports single-speaker lip-sync, two-speaker dialogue with both sides rendered, or idle motion without any audio. All output is standard MP4 video, and you can stitch both sides of a dialogue into a single side-by-side video using a standard video tool called ffmpeg. The project also offers a managed cloud API at avaturn.live if you want to skip the GPU setup entirely. Model weights and inference code are publicly available, a technical report and production-ready backend are listed as coming soon.