rbrown101010/rileyjarvis

RileyJarvis is a desktop AI companion app that runs entirely on your own computer using Electron, the same framework behind apps like Slack and VS Code. It gives you a friendly animated face on screen that you can have a real time, spoken conversation with, similar to talking to a voice assistant, but built specifically around OpenAI's Realtime API for speech to speech conversation. Beyond just talking, the app includes a visual panel where it can show you things while you chat: markdown notes, diagrams, generated images, and progress updates. It can generate images and build a running board of numbered picture versions you can flip back through and edit. It can also search the web if you provide a search API key, and it keeps local notes and records saved on your machine rather than in the cloud. One notable feature is an optional computer control mode. When you turn it on, the assistant can open applications, click buttons, type text, scroll, take screenshots, and inspect what is on your screen, all on macOS. This is opt in and blocked by default until you specifically switch to that mode, since it lets the AI actually operate your computer. To run it you need a Mac, a recent version of Node.js, and an OpenAI API key that has access to the Realtime and image generation features, a search API key is optional. Setup involves cloning the code, installing dependencies with npm, copying an example environment file, and starting it with a single command. The developer notes that certain macOS permissions, like microphone, accessibility, and screen recording, will be requested depending on which features you use. The project keeps your API keys and any data it creates locally on disk, ignored from version control, and the author includes a short reminder to check for accidentally committed secrets before sharing your own copy of the project. It is released under the MIT license.