portkey-ai/gateway

★ 11,710TypeScriptAudience · developerComplexity · 3/5Setup · easy

Mindmap

mindmap
  root((gateway))
    What it does
      AI request routing
      Single interface
      Sub-ms overhead
    Reliability
      Auto retry
      Provider fallback
      Load balancing
    Safety
      Content guardrails
      Response filtering
    Deployment
      Local Node.js
      Docker
      Cloudflare Workers
      AWS GCP Azure

mindmap root((gateway)) What it does AI request routing Single interface Sub-ms overhead Reliability Auto retry Provider fallback Load balancing Safety Content guardrails Response filtering Deployment Local Node.js Docker Cloudflare Workers AWS GCP Azure

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Route your app's AI requests through one endpoint to switch between OpenAI, Anthropic, and Google without changing your code.

USE CASE 2

Configure automatic fallback to a backup model if your primary AI provider goes down.

USE CASE 3

Add guardrails that block unwanted content in AI responses before they reach your users.

USE CASE 4

Deploy the gateway on Cloudflare Workers or AWS for low-latency AI routing in production.

Tech stack

TypeScriptNode.jsDockerCloudflare Workers

Getting it running

Difficulty · easy Time to first run · 5min

In plain English

Portkey AI Gateway is an open source router that sits between your application and AI model providers. Instead of writing separate code to talk to OpenAI, Anthropic, Google, or any other AI service, your application talks to the gateway using a single consistent interface, and the gateway handles the conversation with whichever provider you choose. The gateway supports over 250 AI models from dozens of providers, and connecting to a new one is a matter of changing a few configuration values rather than rewriting integration code. The library claims a latency overhead of under one millisecond, which makes it practical to run in production without noticeably slowing down responses. Beyond routing, the gateway adds reliability and safety features. You can configure it to retry failed requests automatically, fall back to a different model if the primary one is unavailable, and spread load across multiple providers or model instances. Guardrails let you define rules about what the AI is allowed to return, for example blocking responses that contain certain words or topics, and rejecting them before they reach your users. Running the gateway locally takes a single command using Node.js. A built-in console at localhost shows a log of all requests passing through. The gateway can also be deployed on Docker, Cloudflare Workers, or cloud infrastructure on AWS, Azure, and GCP. Enterprise deployments with private hosting and additional access controls are available through Portkey's commercial offering. The project works with popular AI development libraries including Langchain, LlamaIndex, and CrewAI, and is compatible with the standard OpenAI client format so teams already using that interface can switch over without changing their existing code structure.

Copy-paste prompts

Prompt 1

I use OpenAI in my app. Show me how to swap to the Portkey AI Gateway so I can switch between providers without changing my existing code.

Prompt 2

How do I configure Portkey Gateway to automatically retry failed AI requests and fall back to Anthropic if OpenAI is unavailable?

Prompt 3

I want to block certain words from AI responses before they reach users. How do I set up guardrails in Portkey Gateway?

Prompt 4

How do I deploy Portkey AI Gateway on Cloudflare Workers and point my existing OpenAI client at it?

Prompt 5

Show me how to set up load balancing across multiple AI providers in a Portkey Gateway config file.

Open on GitHub → Explain another repo

← portkey-ai on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.