rentheria/llm-budget-cap

{"i": 0, "repo": "Rentheria/llm-budget-cap", "stars": 1, "lang": "TypeScript", "ok": true, "char_count": 2847, "text": "llm-budget-cap is a small TypeScript package that puts a hard spending limit on calls to AI services like OpenAI, Gemini, and Anthropic. The core problem it addresses is real and practical: a bug, a loop, or abuse from many IP addresses can spike an AI bill to thousands of dollars overnight. Existing rate limiters that throttle per IP do not help when an attacker is spread across many addresses. This package provides a single hard counter that cuts off spending once a limit is reached.\n\nThe way it works is simple from the user's side. You create a BudgetCap object with a Redis connection, a key name, and a limit (say, 500 calls per day). Before you make a paid AI API call, you run checkAndIncrement. If the limit is exceeded, the method tells you to block the request. You can also set per-user or per-tenant limits by passing a subKey, and you can configure the time window to any length in milliseconds.\n\nThe technical detail that sets this package apart is how the counter works inside Redis. A naive approach reads the count, checks it, and writes the new count in separate steps. Under high traffic, two requests can both read the same count at the same time, both pass the check, and both proceed, letting one extra call through. To prevent this, the package runs the increment and the expiration timer inside a single Lua script. Redis executes that script start to finish without interruption, so no two requests can slip through at the boundary.\n\nBy default, if Redis goes down, the package lets requests through in a degraded mode rather than blocking your entire feature. You can change this to fail closed if you prefer. The package has zero third-party dependencies beyond the Redis client you already use, and works with Express, Fastify, NestJS, or any other framework. The full README is longer than what was shown.", "error": null}