hkust-c4g/anytalker

AnyTalker is a Python-based tool that takes audio recordings as input and generates realistic talking-head videos featuring one or multiple people speaking. The core problem it solves is creating natural-looking video of people talking, driven purely by audio, without needing to record the actual speakers on camera. The way it works is through an audio-driven framework with a flexible multi-stream structure. You provide a photo of a person (or people) and an audio file, and the system generates video in which the faces move and speak in sync with that audio. It supports both single-person and multi-person scenes, and is designed so that when multiple identities appear together, their interactions look natural and seamless rather than pasted-together. Two model sizes are available, a 1.3B parameter version and a larger 14B parameter version for higher-quality results. You would use this when you want to create a video of someone speaking without filming them, for example, animating a portrait, dubbing content into another language with matching lip movement, or generating presentation videos from a script. The tool is written in Python and relies on audio encoding and video generation components, with FFmpeg needed to produce the final video files.