levantolabs/switchboard-poc

Switchboard is a proof of concept proxy server that automatically picks which AI model should answer each request. Instead of a developer hard coding which model to call, Switchboard sends the incoming prompt to a service called Levanto Sage, which reads the prompt and a short description of what each available model is good at, then chooses the cheapest model that can actually handle the task. The chosen model is then called through OpenRouter, a service that gives access to many different AI models through one API. The project speaks the same request format as OpenAI's API, so an existing app that already uses an OpenAI client library can switch to Switchboard by only changing the web address it points to, with no other code changes needed. The response includes a header showing which model was picked, how confident the routing decision was, and how long the decision took, and there is also a small web based chat playground that shows these decisions and their costs live. The entire routing logic lives in two small files: one that lists the available models along with a plain description of when each should be used, and one that asks Sage a single question to pick the best match. Changing which model handles which kind of task is done by editing that plain description, not by writing new code. If Sage cannot be reached, the system falls back to a fixed default model so requests still succeed. The project is organized into three parts: the core routing logic with no build step, a small server built with the Hono framework that exposes the proxy endpoints, and a Next.js based web playground. The author notes this runs as a single local process only, and that reported cost savings are illustrative rather than a strict benchmark, since a cheaper model can sometimes use more tokens to answer. No license is stated in the README.