taekyungki/avatarforcing

Avatar Forcing is a research project for generating real-time, interactive talking head avatars, digital faces that respond to you in conversation. The problem with existing approaches is that they tend to produce one way animations: the avatar plays out a pre-generated response but does not genuinely react to you in the moment. This project aims to build avatars that respond to both verbal and non-verbal cues, like someone speaking, nodding, or laughing, with low enough latency that it feels like a real exchange. The key idea, called diffusion forcing, is a way to generate avatar motion step by step in real time while respecting the constraint that the system can only see past information, not future input. This lets the avatar react instantly to live audio and motion from the user. The project also introduces a training technique that teaches the avatar to be more expressive without requiring manually labeled data, by having the model compare its behavior with and without user input as a signal for what good reactions look like. The result is a system that runs at around 500 milliseconds of latency and is about 6.8 times faster than the baseline approach it was compared against. In tests, human evaluators preferred its reactive, expressive motion over the baseline more than 80 percent of the time. This is academic research published at CVPR 2026, from researchers at KAIST, NTU Singapore, and DeepAuto.ai. As of the readme, the code for this project has not yet been released, so it is not currently something you can install or run.