bespokeontology/dragon.c

Dragon.c is a hand built inference engine, written from scratch in the C programming language, that gets a huge 560 billion parameter AI model called Meituan LongCat Flash Thinking running on an ordinary laptop with only 31 gigabytes of memory. Normally a model this size, which takes up 531 gigabytes on disk, would need a server with far more memory than that to load it all at once. This project avoids that by reading the model's weight files directly from disk as needed, using the operating system's own memory management to swap pieces in and out automatically, rather than loading everything into memory up front. The author's central claim is that the real bottleneck for running large AI models on normal computers is not the size of the model itself, but how efficiently the computation and data movement are organized. To prove this, every single mathematical step of the engine, from normalization to attention to the mixture of expert layers the model uses, was checked one piece at a time against Python's PyTorch library to make sure the numbers matched almost exactly, before moving on to the next piece. Through four rounds of performance improvements, generation speed increased 50 times over, reaching about 6 seconds per generated word, while producing output identical to the reference implementation the whole way through. Along the way the project uncovered and documented several undocumented technical details needed to correctly reproduce the model's attention mechanism, plus a real bug where a corrupted reference file, not the new C code, was the actual source of a mismatch. This is a research and engineering writeup more than a polished tool, aimed at people working on AI model internals or performance engineering rather than casual users. It requires a machine with a capable GPU, a large amount of RAM, and Linux, plus comfort reading dense technical detail about model architecture and numerical precision. The full README is longer than what was shown.