roalfb/olivia-ai

Olivia is a web application that runs entirely in your browser and pretends to be a physical ESP32 hardware device for Xiaozhi, an open source AI assistant ecosystem normally designed to run on a real microcontroller board with a microphone and speaker. Instead of needing to buy and flash actual hardware, Olivia lets you try out and build against the Xiaozhi cloud using just a browser tab. To pull this off, Olivia has to work around several things a browser normally cannot do that real ESP32 firmware can. Browsers cannot send certain custom authentication headers when opening a WebSocket connection, so Olivia runs a small server component, built with the Hono framework on Cloudflare Workers, that adds those headers on the device's behalf. The same server component also proxies image uploads for the vision feature and handles the device registration process, both of which also require headers a browser cannot set directly. For audio, the Xiaozhi server only accepts properly encoded Opus audio rather than raw microphone data, so Olivia loads a WebAssembly build of the Opus audio codec to encode your microphone input into the exact format the real hardware would send. Because it runs in a browser rather than on a physical chip, Olivia also gains some abilities a real device cannot have, such as switching between a phone's front and back camera, keeping a full text history of conversations, showing a live protocol debug panel, supporting more image formats than a hardware camera would capture, and being reachable from any device with a browser simply by sharing a web link rather than shipping physical hardware. On the AI and conversation side, it connects to the official Xiaozhi cloud service, streams text responses as they are generated, decodes emotion signals sent by the server, and keeps a stored conversation history. On the voice side, it captures microphone audio in real time with noise suppression and echo cancellation, supports both push-to-talk and continuous listening modes, and plays back the AI's spoken responses with minimal delay. The full README is longer than what was shown.