compsmart/talking-face

This project, called Face Cloner, builds a glowing wireframe head in the browser that can talk out loud using Google's Gemini Live voice API, and can also wear the photo of a real face mapped onto its shape. It is written from scratch in plain JavaScript with no frameworks or build tools, using WebGPU for the graphics and WGSL for the shaders. The wireframe is based on a standard 468 point face model from Google's MediaPipe library, which can find those same points in any uploaded photo. Because the points always land in the same places, mapping a photo onto the head is a matter of matching each point in the image to the matching point on the mesh, then cross fading between the glowing wireframe look and the photo using a slider. To use it, a person runs a small bundled server, opens the page in a WebGPU capable browser like Chrome or Edge, and pastes in a free Gemini API key. Typing a message makes the face answer out loud, with lips that move in time with the generated speech, driven by analyzing the incoming audio slightly ahead of when it plays. There is also a feature that scans an uploaded photo, asks Gemini to guess who the person or character might be, and automatically writes a matching speaking style and voice for the face to use, which the user can review or edit before starting a conversation. Under the hood, the model can also send special signals mid conversation to change the face's expression or trigger a gesture, so it can smile or raise an eyebrow while it talks. The project notes that a future version could add microphone input for a full back and forth voice conversation, since the groundwork for that is already in place. This is a hobby scale, single contributor project best suited for developers curious about real time graphics, voice APIs, or expressive avatar experiments, rather than a polished consumer product.