algorithmicsuperintelligence/optillm

OptiLLM is a proxy server that sits between your application and an AI language model API. Instead of sending your question directly to the model, you route it through OptiLLM, which applies various reasoning and optimization techniques before and after the model responds. The goal is to get better answers, particularly on tasks involving math, coding, and logic, without changing the model itself or doing any training. The library implements over 20 techniques. Some are straightforward, like generating multiple responses and picking the best one (called Best of N). Others are more involved, such as running a tree search over possible reasoning paths, using a formal logic solver (Z3) for mathematical problems, or having multiple AI agents cross-check each other's answers. You select a technique by prefixing the model name in your API call, so moa-gpt-4o-mini routes your request through the Mixture of Agents approach. Because OptiLLM exposes the same API shape as OpenAI's service, you change one line of code (the base URL) and your existing application starts using it. It works with OpenAI, Anthropic, Google, and over 100 other models via a routing library called LiteLLM. You can install it via pip, run it with Docker, or build from source. The README presents benchmark results showing meaningful accuracy gains on several standardized tests: for example, a technique called MARS improved a Gemini model's score on a math competition benchmark by 30 points. These numbers come from specific academic benchmarks and reflect the best-case results for each technique. This is a tool for developers and researchers who want to squeeze better performance out of existing models by spending more compute at inference time rather than training a new model. The full README is longer than what was shown.