nasan-duodushu/gateapi

GateAPI is a gateway that sits in front of multiple large language model providers and turns them into a single endpoint. The README describes the situation it is built for: a user who has accumulated API keys from several different LLM resellers, where some keys speak OpenAI's protocol, some speak Anthropic's, and each one supports a different set of models. GateAPI aggregates all of those upstream sources so the downstream caller only needs to talk to one address, normally /v1/chat/completions, and the gateway figures out where the request actually goes. The routing layer does more than dispatch. It tracks rolling latency and error rates per channel over the last fifty calls, and downweights channels that are slow or returning too many errors. A 429 rate-limit response triggers an exponential cooldown from thirty seconds up to five minutes. If an upstream returns an empty response, the gateway detects that and retries on another channel. Both OpenAI and Anthropic protocols are accepted as input and translated to whichever format the upstream expects, including SSE streaming in both directions. The second main feature is a detection engine. It can batch-test every model on every channel for basic reachability and latency, and it can also run thirteen functional probes that check things like math, logic, response length, token accounting, and tokenizer fingerprints. The README explains there is a database of more than fifty tokenizer fingerprints used to guess which model family a channel is actually serving, which is meant to catch resellers that swap a cheaper model in behind a premium name. Management happens through a built-in web admin panel with a dashboard, channel CRUD, key management, request logs, scheduled re-detection, response caching, and Telegram, Discord, or HTTP webhook alerts. The panel supports Chinese and English and has a dark mode. Storage is SQLite. There is also a prompt engine that can inject a system prompt and compress conversation history. Deployment targets are Linux and Docker, Node.js 18 or higher. The default admin password is admin123 and the README warns to change it on first login. License is Apache 2.0.