alantsev/ds4

DwarfStar 4, also called ds4, is a small C based inference engine built specifically to run the DeepSeek V4 Flash AI model on your own computer. Unlike general purpose tools that can load many different model formats, this project intentionally supports only one model, in exchange for very tight and fast integration: it handles loading, prompt formatting, tool calling, and both memory based and on disk storage of the model's working state, plus a server API meant to work with coding agents or the included command line interface. The project targets Apple computers with Metal graphics, particularly MacBooks with 96GB of RAM or more, as its main platform, with NVIDIA CUDA support as a second target and community maintained AMD ROCm support kept on a separate branch. A CPU only mode exists for testing correctness, but it is not meant for regular use. The authors chose DeepSeek V4 Flash because it runs faster than other models of similar strength, produces shorter reasoning sections that scale with how hard the question actually is, supports a context window of one million tokens, and compresses its working memory well enough that long conversations can be saved to disk instead of staying only in RAM. The project also provides tools for creating and quantizing the special model files it needs, along with speed and quality testing utilities. Documentation covers testing for contributors, generating quantized model files, collecting calibration data, and running benchmarks. The engine only works with model files built specifically for it, not arbitrary DeepSeek or generic model files. The project is built with heavy use of AI coding assistance, is still labeled alpha quality software, and gives credit to the llama.cpp project for the groundwork it builds on. The full README is longer than what was shown.