victorsteinbock/llama-cag-n8n-reworked

llama-cag-n8n-reworked lets you ask questions about a document using a fully local AI model, without that model having to reread the whole document every time you ask something new. The idea it's built around is called Cache-Augmented Generation, or CAG. Normally, an AI model either has to process your entire document again for every question, which is slow, or it only sees small retrieved snippets from the document through a technique called RAG, which can miss connections between different parts of the text. This project takes a different path. You hand it a document once, and the model reads the whole thing a single time on your own computer. As it reads, it saves its internal understanding of the text, called the KV cache, to disk. From then on, every question you ask is answered by restoring that saved understanding rather than reading the document again, so answers come back in seconds instead of minutes, at no ongoing cost, and without any of the document leaving your machine. Everything runs inside Docker containers, so there's nothing to compile by hand on your own computer. Under the hood, llama.cpp's llama-server handles running the AI model and keeping the cache, a small FastAPI service coordinates the pieces, n8n handles automation and workflow logic, and PostgreSQL stores metadata. It's designed to work on Windows, macOS, or Linux. The README describes several situations where this approach works especially well: a support chatbot that needs to answer strictly from a real product manual instead of guessing, a way to let AI coding assistants like Claude Code ask questions about a large specification document without that document eating up their limited context window every session, and a private reference desk for a team's contracts, runbooks, or compliance documents that can't leave the building. It's also honest about where it doesn't fit. It's not meant for searching across thousands of documents at once, for documents too large to fit in the model's memory, for multi-user login systems, or for documents you'll only ever ask about once. The document needs to be readable as text, so scanned or image-based PDFs need an extra preparation step first. The local server also runs unauthenticated, relying on being reachable only from your own machine rather than the open internet. The project is released under the MIT license. The full README is longer than what was shown.