llmrelay/relay

relay is a self hosted gateway that sits between your applications and a range of AI providers, letting you talk to any of them through one server on your own machine. It accepts requests written in either the OpenAI or Anthropic API formats, so tools built for one of those, such as Claude Code or any OpenAI compatible SDK, can be pointed at relay instead, and it forwards those requests out to whichever provider you have configured, including OpenAI, Anthropic, Gemini, Groq, DeepSeek, Mistral, or a locally running Ollama model. It ships as a single small program with no external dependencies to install, and it does not collect or send any usage data back to its maker. Getting started involves installing the program, setting environment variables for whichever provider API keys you already have, and starting the server, after which any tool that speaks the OpenAI or Anthropic protocol can be redirected to it with a single URL change. A setup command can generate a starter configuration file that documents where to find each provider's key. Routing between providers can be as simple as fixed rules choosing the cheapest, fastest, or a specific model, with built in reliability features like automatic retries, spreading requests across a pool of API keys, and switching away from a failing provider mid stream. There is also an optional smart routing mode that tries to send easy requests to a cheap model and harder requests to a stronger one, but this mode ships turned off by default. The project includes its own built in evaluation tool for testing whether that smart routing genuinely beats simply always using a strong model on real traffic, and its own published results show one of its routing tiers currently failing that bar, which is why it stays off unless a user turns it on deliberately. The project is written in Go, distributed as a single static binary, and released under the Apache 2.0 license, which allows free use, modification, and redistribution including for commercial purposes.