one-million-lines/privacy-pii-redactor

Privacy-First PII Redactor is a tool that sits between your application and an external AI service. Before your prompts leave your system, the tool scans them for private information, replaces each piece with a labeled placeholder like EMAIL_1 or CREDIT_CARD_1, and forwards the cleaned version. Optionally, it stores a mapping and restores the original values into whatever the AI sends back. The detection system combines three layers. Regular expressions catch clearly structured data: email addresses, phone numbers, credit card numbers (with Luhn validation), IBAN bank account numbers, social security numbers, IP addresses, and more. Microsoft Presidio (a Microsoft open-source library) and spaCy (a natural language processing library) add a second and third layer that can recognize names, organization names, and locations even without predictable formatting. When the three layers detect overlapping matches, the tool resolves conflicts by priority: regex wins over Presidio, which wins over spaCy. You can use this tool three ways. As a Python library, you import it and call redact() on any string, getting back both the cleaned text and a mapping of what was replaced. As a command-line tool, you can process files or strings from the terminal. As a REST API running locally via Docker, you send text to a /v1/redact endpoint and receive cleaned text in return. There is also an LLM proxy mode where the tool acts as an OpenAI-compatible endpoint: it redacts your request, forwards it to the actual AI provider, and puts original values back into the response before returning it to you. The README includes an important warning: detection is probabilistic. Names and physical addresses are harder to catch reliably than emails or credit card numbers. The tool reduces risk but does not guarantee compliance with privacy regulations like GDPR or HIPAA. The license is MIT, allowing free use including commercial applications.