omarish/llm-sql

llm-sql is a project that runs GPT-2, a small text generating AI model, entirely inside a Postgres database using plain SQL. Every part of how the model produces text, including looking up word pieces, tracking position in a sentence, its internal math layers, and even the process that breaks input text into tokens, is written as SQL functions instead of the usual Python or specialized AI code. The model's weights, which are the numbers the AI learned during training, are stored as regular database rows using a Postgres extension called pgvector that is built for storing and comparing lists of numbers efficiently. When you type a SQL command like asking it to complete the phrase "Hello, I", the database itself performs all the calculations needed to guess what text comes next, one word piece at a time. This is explicitly a toy and learning project rather than a fast or production ready tool. Generating each word piece takes about one second, since the database is doing work that would normally run on specialized AI hardware. The project does include a caching trick so that generating a new word only requires looking at the newest input rather than reprocessing the whole conversation each time, which keeps it from getting slower as the text grows. To try it, you need Docker to run the Postgres database, a Python tool called uv to manage dependencies, and the psql command line client. The setup process downloads the GPT-2 model, exports its weights into files, loads them into the database, and installs the SQL functions. A demo script then lets you type a prompt and see the generated text. The author is upfront that this should never be used anywhere exposed to the internet, since the included database setup disables normal safety checks and uses simple default passwords, only meant for a private test on your own machine. It only supports the smallest version of GPT-2, and its word splitting logic is a close but not perfect match for the original.