amitshekhariitbhu/transformers-explained

transformers-explained is a curated collection of blog posts that walk through the Transformer architecture, the design behind almost every modern large language model, one concept at a time. It is written and maintained by the founder of Outcome School and is meant to be read rather than run, since the repository is mostly a set of links pointing to articles hosted on the author's blog and Substack. The series is organized into parts. The first part covers the full architecture end to end, explaining tokenization, embeddings, positional encoding, the attention mechanism, feed forward networks, residual connections, layer normalization, and how the encoder and decoder pieces fit together. It also touches on the different variants of the Transformer and why the design became so influential. The second part is dedicated entirely to attention. It starts with an intuitive explanation of self attention, then moves into the underlying math of query, key, and value vectors, including a worked numeric example. From there it covers why attention scores are scaled, how multi head attention combines several attention calculations, and how cross attention differs from self attention. Later posts explain causal masking, which stops a model from seeing future tokens, grouped query and multi query attention as ways to make attention cheaper to run, and Flash Attention, a technique for making attention faster on GPUs by managing memory more carefully. The third part focuses on position. Since attention alone treats a sentence like an unordered set of tokens, this section explains positional embeddings, the technique that tells a Transformer which token came first, including the math behind Rotary Position Embedding, a widely used method in current models. The project has no listed programming language because it contains no runnable code. Instead it functions as a reference guide for people who want to understand how Transformers work internally, useful for students, researchers, or engineers building on top of large language models. The author notes that the series will keep growing as new blog posts and videos are added.