gradium-ai/gradbot

Gradbot is an open-source framework for building voice agents, meaning software that can hold spoken conversations with people in real time. The core problem it addresses is that connecting a microphone input to an AI language model and back to a speaker involves coordinating three independent streams simultaneously: speech-to-text (converting what the user says to words), the AI model generating a response, and text-to-speech (converting the AI's words back to audio). Handling interruptions, turn-taking, and silences across all three is complex plumbing that Gradbot handles for you. The engine is written in Rust for performance and runs a state machine that moves between listening, processing, and speaking states. You write the conversational logic and personality in Python (roughly 50 lines for a basic agent), while the framework manages the audio coordination underneath. It works with any OpenAI-compatible language model, such as GPT-4o, Claude, or locally-run models via Ollama, and uses the Gradium service for streaming speech recognition and text-to-speech across 5 languages. It also supports tool calling, meaning your voice agent can trigger functions in the background while still talking naturally, and integrates with MCP servers (a standard way to give AI systems access to external capabilities). You would use this to quickly prototype voice-driven experiences: customer support agents, booking assistants, game characters that speak, educational tutors, or any interactive application where users talk rather than type. It is available as a Python package and runs demos as FastAPI applications.