stfurkan/bitgpu

bitgpu is a runtime that lets a web browser run certain small AI language models directly on your own graphics card, using a browser technology called WebGPU. It currently supports a family of models called Bonsai that store their internal numbers in a very compact one bit format, which keeps the download size and memory use low compared to typical models. The project ships ready made model files, so you can point your code at a hosted model and start chatting without converting anything yourself or running a server. It handles the full generation loop: picking the next word over and over, streaming each piece back to you as it is produced, and stopping cleanly when the model finishes or you cancel it. It also supports several ways of choosing which word comes next, including plain greedy selection and more varied sampling methods that reduce repetition or keep answers closer to the most likely options. Conversations can be saved and reloaded across page refreshes without having to reprocess everything from scratch. A built in benchmark compares bitgpu's speed against another popular browser based runtime on the same model and hardware, and the project reports meaningfully faster response times in that comparison, though it notes results will vary by device and browser version. The library has zero runtime dependencies once installed, and automatically falls back to a slower but more compatible code path on graphics hardware that lacks certain fast features, so it keeps working across a wider range of devices. This is aimed at developers who want to add offline capable, privacy friendly AI chat features to a web page without needing a backend server or sending user data anywhere.