emmimal/control-layer

This Python library adds a reliability and safety layer between your application and any large language model. Most LLM integrations send a prompt and use whatever comes back. This library handles everything that can go wrong in production between those two steps: blocking bad input before it reaches the model, making sure the output matches what your application actually needs, recovering when the model fails or returns garbage, and keeping a record of every attempt. The library works as a pipeline of components you can configure and combine. First, an input guard checks the user's text for prompt injection patterns, rejects inputs that are too long, and sanitizes what passes through. A token budget component counts tokens accurately and allocates space across your system prompt, constraints, context, and user input so the assembled prompt never silently overflows the model's limit. A circuit breaker tracks consecutive failures and stops sending requests during a recovery window rather than hammering a backend that is down. After a response arrives, a validator checks it against a schema: is it valid JSON, does it contain the required keys, is it within length bounds, does it contain any forbidden phrases. If validation fails, a retry engine maps the specific failure type to a targeted change in the prompt and tries again with jittered exponential backoff. If retries are exhausted, a fallback router calls a registered chain of alternatives, such as a cached response or a simpler template, and returns the first one that produces a result. Every attempt is written to a JSONL audit log with timing and outcome data. The whole thing accepts any LLM as a function that takes a string and returns a string, so you can use it with OpenAI, Anthropic, a local model, or a mock during testing. No machine learning dependencies are required and no GPU is needed. The four required packages are tiktoken for token counting, pydantic for configuration validation, tenacity for retry logic, and structlog for structured logging. The repository includes five runnable demos that cover each failure mode and recovery path using a mock LLM that simulates realistic failure rates, and 69 tests covering all components individually and in combination.