myned-ai/nyxclaw

NyxClaw is a self-hosted server, built in Rust, that gives any AI agent a real-time animated avatar with lip-sync and voice. You run it on your own machine alongside your AI agent, and it connects to a companion mobile app that shows the avatar's face. When the AI speaks, the avatar's mouth moves in sync, it also keeps breathing and blinking during processing gaps rather than freezing. The animation system works by converting audio into 52 ARKit blendshapes, a standard format for facial animation used in phone cameras, at 30 frames per second, all on CPU with no GPU required. You can interrupt the avatar mid-sentence and both the AI and the audio playback cancel within about 128 milliseconds. Two voice pipelines are available. The OpenAI Voice pipeline uses OpenAI's Realtime API for speech-to-text and text-to-speech, so audio goes through OpenAI's servers. The Local Voice pipeline uses faster-whisper with Silero VAD for speech recognition and Piper VITS ONNX for speech synthesis, so nothing leaves your machine. Privacy and security are central to the design. The server pairs with your phone via QR code using Ed25519 cryptographic authentication, and the connection travels over an automatically provisioned encrypted WebSocket tunnel with no port forwarding or certificates needed. There is no cloud relay and no telemetry. NyxClaw currently supports two AI agent backends, OpenClaw and ZeroClaw, with optional patches that enable richer behavior such as splitting responses into spoken audio and visual content shown on the phone. The mobile app is available on Google Play. The full README is longer than what was provided.