0hq/webgpt

WebGPT is a proof-of-concept project that runs a GPT-style language model directly inside a web browser, without any server sending the computations off to a cloud. It uses WebGPU, a newer browser technology that gives web pages access to the computer's graphics card for heavy number-crunching tasks. The entire implementation is written in plain JavaScript and HTML, coming in at under 1500 lines of code. The project includes two pre-packaged models you can try: a small model trained on Shakespeare's texts (described by the author as undertrained) and a larger GPT-2 model with 117 million parameters. Benchmarks on an M1 Mac show the 117M model generating text at around 30 milliseconds per token, meaning you get a new word roughly every 30 milliseconds. Running it locally is straightforward because it is just HTML and JavaScript files. The main requirement is a browser that supports WebGPU, which was still rolling out to major browsers when this was written. Chrome Canary or Edge Canary are the recommended options. The model weight files are large and stored using Git LFS, a system for tracking big files in a code repository, so you need that installed to download them after cloning. The project was built as a learning exercise. The author started with no background in how transformer models, GPU programming, or text tokenization work, and credits Andrej Karpathy's public video series for the foundational understanding. Code from the nanoGPT project and a JavaScript GPT tokenizer were also used as references. The roadmap in the README lists several remaining optimizations and open questions the author plans to address.