prayangshuuu/hummingbird

Hummingbird is a program written in the C programming language that lets you run very large artificial intelligence models on ordinary computers. Large AI models, some with hundreds of billions of parameters, normally require expensive hardware with enormous amounts of memory. Hummingbird tackles this problem by spreading the model across the storage your computer already has, including your solid state drive, system memory, and graphics card memory. It treats all three as one connected pool. The project is inspired by an earlier project called Colibri, which could run a 744 billion parameter model on a machine with 25 gigabytes of RAM by streaming data from disk. Hummingbird takes a different approach. Instead of being built for one specific model, it is designed as a general framework that can run many different model architectures through a flexible adapter system. It supports models like GPT-OSS 120B, DeepSeek, Qwen, and GLM. The core idea is a memory hierarchy. Weights the model needs for every single token stay resident in fast memory. The much larger pool of expert weights lives on disk and is pulled up only when needed. A learning cache keeps frequently used weights warm, so the engine gets faster with repeated use. Expert weights are read in one large sequential chunk and handed directly to computation kernels without being copied. You can install Hummingbird on Linux or macOS with a single command line script. Alternatively, you can build it from source with a C17 compiler and CMake. There are no third party runtime dependencies, no BLAS, and no Python required. The build system lets you toggle options like a CPU backend, a CUDA backend for NVIDIA GPUs, or a Metal backend for Apple Silicon. The command line tool, called hb, supports one shot text generation, interactive chat mode, and an OpenAI compatible HTTP server. The project is still in an early foundation stage. Inference is currently in development, and upcoming features include a model loader, dynamic batching, and a streaming engine. You can also embed Hummingbird into your own application using its stable C interface.