rudysev/host-assistant

This project is a starting point for running a voice assistant locally on a home network, instead of relying on a cloud service. It is meant to work alongside a separate app called portal-assistant, which normally sends voice data to Google's Gemini cloud service to understand speech and generate a spoken response. This project lets that same app talk to a computer on the user's own local network instead, so someone can demo, test, or build on the underlying pipeline without depending on that cloud service. When someone speaks into the portal-assistant app, the audio is streamed over the local network to this project, which detects when the person has finished talking, converts their speech into text, sends that text to a locally running language model that can also call a small set of tools, such as checking the weather or searching the web, and then converts the model's reply back into spoken audio that streams back to the app in real time. Any tool the model needs that is not handled locally is passed back to the portal-assistant app itself to execute. The project stitches together several existing open source pieces rather than building everything from scratch: a speech to text engine, a voice activity detector to know when someone has stopped speaking, a locally run large language model server, and a text to speech engine. As shipped, part of that pipeline, the speech to text step, is built specifically for Apple Silicon Mac computers, though the README notes it can be swapped for another version to run on Linux. Setting it up involves installing a local language model runner, creating a Python virtual environment, installing the project's dependencies, and configuring a few settings such as which web search service to use. Once running, it listens for a single connection at a time from the portal-assistant app over the same Wi-Fi network. The project explicitly describes itself as a demo rather than a finished, polished product, and notes it is an independent, unofficial project not affiliated with Meta, whose Portal hardware the companion app is designed for. It is released under the MIT license.