cloud-ru-tech/guardrails-llm-filter-extproc

guardrails-llm-filter-extproc is a Go based service that protects sensitive information as it flows between an application and a large language model. It works alongside Envoy, a popular proxy server, using a feature called an external processor. Traffic still passes through Envoy as normal, but Envoy hands the request and response bodies to this filter over gRPC so it can inspect and clean them on the way through. The filter's job is to find things like passwords, API keys, access tokens, IP addresses, and personal information, including Russian identifiers such as SNILS, INN, and OGRN numbers, bank card numbers, and IBANs, checked against their known formats. It uses around 260 built in pattern rules to spot this kind of data inside a request before it reaches the AI provider, and replaces each sensitive value with a safe placeholder, such as a labeled stand in for an email address. When the AI's response comes back, the filter swaps the placeholders back to the real values before the client ever sees them, so from the client's point of view nothing changes, even for streaming, token by token responses. The project already understands the request and response formats used by OpenAI and Anthropic style APIs, including chat completions, messages, and other common endpoints, both regular and streaming. If the filter itself becomes unavailable, Envoy is configured to let traffic through rather than block it, so a failure in the filter does not take down the whole system. There is also a quiet observation mode that only records what it would have masked, useful for testing new rules without changing real traffic. Sensitive data mappings can be stored in memory, in Redis, or in Postgres, optionally encrypted, which allows the filter to work correctly even when multiple copies of it are running behind the same proxy. The project also reports metrics compatible with Prometheus and includes a Grafana dashboard and an audit log for tracking what has been detected over time. It is released under the Apache 2.0 license and is built with Go.