abhisumatk/epstein_files_rag

Epstein Files RAG Explorer is a Python application that lets you search and ask questions about the unsealed Jeffrey Epstein court documents. It uses a technique called Retrieval-Augmented Generation, where an AI model reads relevant passages from a large document collection and uses those passages to answer your questions, rather than relying on general knowledge. The interface is a web dashboard you run on your own computer. The document dataset comes from a public collection on Hugging Face and is stored in a format called Parquet. Because the full dataset is over 200 GB, the setup script by default downloads only the first 0.5 GB chunk, which is enough for testing. You can increase that amount by changing one number in the ingestion script. Once downloaded, the documents are indexed into ChromaDB, a local vector database that makes it fast to find relevant passages given a question. For the AI model that generates answers, the app supports two options: running a model locally on your own machine using a tool called Ollama, or connecting to a cloud inference service like Groq or OpenRouter using an API key. The README notes that Groq in particular is fast. The app is built with LangChain to coordinate the retrieval and generation steps, and Streamlit to provide the browser-based interface. A key design choice is that the assistant is restricted to the Epstein documents. System prompts are configured to refuse questions outside that scope, so the tool stays focused on the investigative context rather than acting as a general-purpose chatbot. The project is open-source under the MIT license.