123satyajeet123/bitnet-server

bitnet-server lets you run an AI language model called BitNet b1.58 2B4T entirely on your CPU, with no graphics card needed. It wraps that model as a local server that speaks the same API format used by OpenAI, so any app or script already built to talk to a chat API can switch over without code changes. The clever part is how it manages memory. When nothing is asking it questions, the server idles at just 112 KB of RAM, about the footprint of a small text file. It achieves this through systemd socket activation, a Linux feature that keeps only a tiny listener process alive. The moment a request arrives, the full model loads, using around 1.4 GB, handles the conversation, then stays ready for follow up requests rather than shutting down immediately. Install is a single apt install command through a package repository, and the roughly 1.2 GB model file downloads automatically on the first request. You interact with it the same way you would any chat API: send a JSON message to a local port and get a response back. Configuration such as port, thread count, and context window size lives in a plain text file and takes effect after a service restart. The project compares itself to Ollama, noting that Ollama does not support BitNet's specialized inference approach, while this package does. It targets Ubuntu 24.04 on amd64 hardware, requires 2 GB of RAM and 1.5 GB of disk space, and is licensed under MIT, building on Microsoft's own BitNet.cpp and BitNet b1.58 2B4T model, both also MIT licensed.