seraphimserapis/ds4

DwarfStar 4, or ds4, is a small native inference engine built specifically to run the DeepSeek V4 Flash AI model on local machines. Unlike general purpose tools that can load many different model formats, this project is intentionally narrow: it only works with the specific GGUF model files the authors publish for DeepSeek V4 Flash, using a graph executor tuned specifically for that model on Apple Metal for macOS or CUDA for Linux. A basic CPU path exists only for correctness checks, not for real use, and the authors warn that current macOS versions have a bug that can crash the CPU path. The README explains why the authors think DeepSeek V4 Flash deserves its own engine rather than being treated as one of many supported models. Compared to smaller dense models, they say it runs faster because it activates fewer parameters at once, its reasoning steps stay proportional to how hard a problem actually is, it supports a one million token context window, and its key-value cache compresses well enough to persist to disk instead of staying entirely in memory. They also note it performs well when quantized down to 2 bits, which lets it run on a MacBook with 128GB of RAM. To use it, a person downloads one of the provided quantized model files with a script, choosing a smaller version for 128GB machines or a larger one for machines with 256GB or more, then builds the project with make. This produces two programs: a command line tool and a server with an HTTP API, both of which only work with the specific model files this project ships. The README is explicit that the project was built with heavy use of an AI coding assistant, with humans directing the ideas and doing the testing and debugging, and it credits the llama.cpp and GGML projects for the kernels, formats, and engineering knowledge its own code builds on. The authors describe the current code and model files as alpha quality, expecting it to take months to become more stable, and the project is licensed under MIT.