alibaba-quark/liveavatar

Live Avatar is a research project from Alibaba Group and academic collaborators that generates streaming, real time avatar video driven by audio, with no fixed limit on how long the generated video can run. It pairs a fourteen billion parameter diffusion model with a system designed specifically for streaming, reaching forty five frames per second on multiple H800 GPUs using four step sampling and a block wise autoregressive generation approach that lets the video keep extending instead of being capped at a short clip. The project describes itself through three main strengths: real time streaming interaction with low latency, autoregressive generation that can sustain video beyond ten thousand seconds, and strong generalization across cartoon characters, singing performances, and other varied scenarios beyond realistic human faces. A recent update added FP8 quantization, letting the model run on GPUs with forty eight gigabytes of memory instead of requiring larger hardware, along with compiler and attention optimizations that further increased speed. Setup starts with a Conda environment running Python, followed by installing PyTorch with CUDA support, an attention library appropriate to the GPU generation being used, remaining Python requirements, and FFMPEG. Two pretrained components are required: a fourteen billion parameter base video model and the project's own smaller model built on top of it, both hosted on Hugging Face and downloaded into a local checkpoint folder. The project offers both multi GPU real time inference for the fastest results and a single GPU inference path for anyone without access to five H800 cards, needing only a single GPU with eighty gigabytes of memory. Planned but not yet released work includes a dedicated interface for streaming interaction, text to speech integration, and training code, meaning the current release is inference only. The code accompanies a published research paper and is intended for people exploring or building on real time avatar generation research rather than for casual use without a capable GPU.