veteranbv/aicrawl

aicrawl is a command-line tool written in Go for building a private local archive of your AI conversation history. It imports official export files from Claude and ChatGPT, stores the conversations in a local SQLite database, and provides search and export capabilities over the stored data. It does not call any cloud API during import or search. The import process accepts the ZIP or JSON export files that Claude and ChatGPT provide through their official export features. Once imported, conversations and messages are stored in the database along with the original raw JSON payloads. Message content is indexed using FTS5, which is SQLite's built-in full-text search system, so you can search across all your conversations with a keyword query and filter or group results by conversation. Beyond basic search, the tool supports retrieving messages around a specific message ID for context, exporting entire conversations or the full archive to Markdown files, and running arbitrary read-only SQL queries against the database directly. A set of structured output commands for automation returns JSON, including metadata, status, and a diagnostic check. The tool is explicit about what it does not do in version 0.1. It does not scrape session tokens, automate browser interactions, support embeddings or semantic search, or provide a graphical interface. It does not sync to the cloud, run as a background daemon, or import data back into the AI services. The project requires Go 1.26.2 or newer to build, which the README notes is required by a dependency called crawlkit. Export files and the generated archive database contain private conversation data, and the README advises keeping them in paths excluded from version control.