mitkox/ds4-zgx-gb10

DwarfStar is a small, purpose built inference engine, meaning software that runs an AI language model and produces its responses, built specifically to run DeepSeek V4 Flash and, on machines with a lot of memory, DeepSeek V4 PRO. Unlike general purpose tools that can load many different models, this project deliberately supports only these specific models, handling their loading, prompt formatting, tool calling, and memory state in a way tailored to how they work. It supports three types of computer hardware: Apple computers with Metal graphics starting around 96 gigabytes of memory, NVIDIA GPUs including systems called DGX Spark, and AMD based systems using a chip design called Strix Halo, found in machines like the Framework Desktop. The project openly credits an existing project called llama.cpp and its GGML library as the foundation that made this possible, even though DwarfStar does not directly link against that library, and keeps its original creators credited under the MIT license. A key design idea here is treating a model's working memory, called the key value cache, as something that can live on a fast solid state drive rather than only in RAM. This means someone with less memory than a model normally requires can still run it, trading some speed for flexibility, instead of simply being unable to run it at all. The README states plainly that this project was built with heavy assistance from an AI system alongside human developers leading the design, testing, and debugging, and says people who are uncomfortable with AI assisted code should not use it. It also warns that the code is beta quality, since the project is only a few days old at the time of writing, and that a companion coding agent feature is even earlier, alpha quality. There is a known bug where running the plain CPU code path on current macOS versions can crash the operating system. This tool only works with specially prepared model files matching DeepSeek V4 Flash or PRO, not arbitrary model files. The full README is longer than what was shown.