arahim3/mlx-dspark

mlx-dspark is a Python library that speeds up text generation from language models on Apple Silicon Macs. It runs a technique called speculative decoding: a smaller "drafter" model proposes several tokens at once, and the main model verifies them all in parallel. When the drafter's guesses are correct, the result is the same as normal decoding but faster. When they are wrong, the main model corrects them. The output is always identical to what the main model would have produced on its own. The library supports two drafter types. DSpark, published by DeepSeek, proposes a block of seven tokens at once using a small parallel network trained to predict the main model's next moves. DFlash, from z-lab, uses a different approach called block diffusion, denoising a full sixteen-token block in one parallel pass. Both work with the Qwen3 and Gemma-4 model families and produce speedups of roughly 1.5 to 2 times on typical text, with larger gains on structured output like code. Installation is a single pip command. The library downloads the main model and a matched drafter from Hugging Face on first use. It runs on Apple Silicon via MLX, no server or external GPU is needed. It can be used from the command line, called from Python code, or run as an OpenAI-compatible API server that tools like LM Studio can connect to. The built-in API server supports streaming, multi-turn chat, tool calling, and prefix caching. Prefix caching keeps the conversation context in memory between turns so follow-up messages do not re-process the full history each time, which speeds up long conversations significantly. The library is MIT-licensed. Matched drafters are available for Qwen3 (4B, 8B) and Gemma-4 (12B), other DSpark or DFlash checkpoints can be used by specifying a drafter path manually.