promtengineer/localgpt

LocalGPT is a tool for asking questions about your own documents using AI, but with everything running on your own computer so no data ever leaves your machine. The README describes it as a fully private, on-premise Document Intelligence platform, you point it at files like PDFs, ask questions in plain English, and it summarises, retrieves, and answers using a local language model rather than a cloud service. The project's description states that data stays on the device and the system is 100% private. Under the hood, it uses a technique called RAG (Retrieval-Augmented Generation), where the AI first looks up relevant passages from your documents and then uses them to compose an answer. LocalGPT extends that with a hybrid search engine that blends semantic similarity, keyword matching, and a method called Late Chunking for long-context precision. A "smart router" decides per query whether to fall back to direct LLM answering or use retrieval, contextual enrichment and sentence-level Context Pruning aim to surface only the most relevant content, and a separate verification pass is run for accuracy. The README lists features like source attribution on every answer, query decomposition into sub-questions, semantic caching with similarity matching, and session-aware chat history. The system runs models through Ollama for inference and uses HuggingFace components for embeddings and reranking, and the README notes support for CUDA GPUs, CPU, Intel Gaudi (HPU), and Apple MPS. You would use it if you want to chat with sensitive documents, internal company files, legal contracts, research papers, personal notes, without sending them to a third party. Installation is by cloning the repository and either running it directly (Python 3.8+, Node.js 16+, plus Ollama for the local model) or using the provided Docker setup, with both flows documented step by step. The core RAG system is written in pure Python with minimal dependencies, and a web UI runs at localhost:3000 once the launcher script is started. The README notes the installation has currently only been tested on macOS.