othersideai/tinygpt

minGPT is a teaching tool that shows you how GPT-style AI models actually work under the hood. Instead of dealing with thousands of lines of production code, you get the whole model in roughly 300 lines. The point is to make the internals of something like GPT-3 readable and hackable rather than mysterious. At a high level, the model takes a sequence of numbers (which could represent words, characters, or even image pixels) and predicts what number should come next. You feed it some context, and it spits out a probability distribution over possible next values. That prediction loop, repeated over and over, is what lets these models generate text or complete patterns. The repo packages this into two core files: one defines the model, and the other handles the training loop. Beyond that, a handful of Jupyter notebooks demonstrate the concept with hands-on examples. The notebooks are where it gets concrete. One trains the model to do addition. Another builds a character-level language model that learns patterns in text. A third shows that the same approach works on images, treating pixel data as just another sequence of integers. These examples are designed for people who learn best by tinkering, students, hobbyists, or anyone who wants to build intuition for how transformers function. The README is careful to note what this project is and isn't. With some additional engineering work, it could potentially reproduce GPT-1 or GPT-2 scale results, but nobody has tested that. GPT-3 scale is out of reach because the model is too large to fit on a single GPU. The project trades power and polish for clarity, which makes it a great starting point if you want to understand the mechanics rather than deploy a production system.