nikhilachale/gwen

Gwen is a voice activated AI desktop assistant that runs as a local Electron application on macOS. You speak to it, it transcribes what you said, sends your request to an AI model that figures out what to do, and then speaks the answer back through a synthesized voice, all without opening a browser or typing anything. Under the hood, three services handle the interaction: OpenAI's Whisper converts your voice to text, Claude, made by Anthropic, acts as the brain that understands your request and decides which actions to take, and ElevenLabs converts the text response back into natural sounding speech. The app also shows a visual orb built with Three.js that changes color depending on Gwen's state: listening, thinking, speaking, or idle. The range of things Gwen can do is wide. On the productivity side it can read your macOS Calendar, check unread Gmail, manage local tasks and notes, and set timers or alarms using natural language. For system control it can adjust volume and brightness, toggle Wi-Fi and Bluetooth, switch dark mode, lock or sleep the machine, and send iMessages or WhatsApp messages, always confirming before it sends a message. It can also search the web, describe what is currently on your screen, open apps and files by name, place FaceTime or phone calls through a paired iPhone, get directions, and control music playback. A shortcuts bridge lets it trigger any macOS Shortcut by name, which extends it to HomeKit and other automations without writing new code. Gwen remembers preferences and facts about you in a local SQLite database between sessions. The project is built in TypeScript with Electron, using Vite and React for the interface and Three.js for the orb animation. To set it up, you install system dependencies like sox, install npm packages, fill in API keys for Anthropic, OpenAI, and ElevenLabs in an env file, and optionally connect Gmail through OAuth. It is licensed under MIT.