Analysis updated 2026-05-18
Download the pretrained earthquake-science language model and run inference immediately
Reproduce the full pretraining pipeline from data crawling through to a trained model
Learn how tokenizer training, model architecture, and multi-GPU training fit together in practice
Study how mixing general text with domain text affects a language model's fluency
| jiazhe868/nanogpt-seis | a-shojaei/constructdrawingai | alex72-py/aria-termux | |
|---|---|---|---|
| Stars | 20 | 20 | 20 |
| Language | Python | Python | Python |
| Setup difficulty | hard | moderate | moderate |
| Complexity | 4/5 | 4/5 | 2/5 |
| Audience | researcher | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
Full reproduction requires 2 GPUs (or one with a smaller batch size), a matching CUDA-enabled PyTorch build, and hours of crawling plus training time.
nanoGPT-Seis is a teaching project that walks through every step of building a small AI language model from scratch, using earthquake science as the subject matter. Rather than trying to build the best possible earthquake expert model, its real goal is to make the whole process of training a language model understandable: where the training text comes from, how it gets cleaned up, how the vocabulary the model uses gets built, why the model's internal design looks the way it does, how training is split across two graphics cards, and how the finished model answers questions afterward. The training material blends earthquake and seismology writing, pulled from open access research papers, preprint archives, and a science newsletter, together with general purpose text from Wikipedia and a broader web text collection, so the model can speak plainly and not just in dense scientific language. This mix adds up to roughly 823 million pieces of text called tokens, and the resulting model has about 113 million adjustable parameters, small by modern AI standards but large enough to become genuinely fluent when trained this way. The project reports its actual measured results rather than just claims, including how much faster and more coherent the model becomes when the training text mix includes general writing alongside earthquake papers, and how giving the model a longer view of surrounding text noticeably improves its predictions. Training took about six and a half hours on two specific graphics cards, though the author notes it can also run on a single more common consumer graphics card with a smaller batch size. A pretrained version of the model is available for anyone to download and try immediately, and the repository also includes every script needed to reproduce the entire process from the very first data download through to running the finished model. The author includes a specific troubleshooting warning about a common graphics driver mismatch that can silently cause the whole training run to happen only on the computer's regular processor rather than the intended graphics card, which would be much slower. The full README is longer than what was shown.
A teaching project that trains a small 113M parameter AI language model on earthquake science texts, explaining every pretraining step from data collection to inference.
Mainly Python. The stack also includes Python, PyTorch, CUDA.
License not stated in the available metadata.
Setup difficulty is rated hard, with roughly 1h+ to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
Verify against the repo before relying on details.