lukallm/dflash_qwen3.6_27b_llamacpp

This repository is a benchmarking project rather than an application you install and run day to day. It measures a technique called DFlash speculative decoding, applied to a large language model called Qwen3.6-27B running through llama.cpp, a popular tool for running AI models efficiently on local hardware. Speculative decoding works by having a smaller draft process guess several upcoming words at once, which the main model then checks, and when the guesses are right this can make text generation much faster without changing the actual output. This project's goal is to answer two questions: how much faster does DFlash make things, and does that speed come at any cost to accuracy. All tests were run on a single high end GPU with 97 gigabytes of memory, enough to hold both the main model and the smaller draft model at once. The results show large speed gains, up to roughly three and a half times faster generation on a 500 problem math benchmark, and even bigger gains, up to about eight times, on longer text generation when combined with an additional n-gram based technique, though the authors note that extreme number is a best case scenario rather than typical everyday performance. Accuracy across math and coding benchmarks stayed within about one percent of the non sped up baseline, which the authors attribute to normal floating point noise rather than a real loss in quality. The repository includes the raw benchmark data as spreadsheet style files, a Python plotting script to regenerate the charts, and a detailed setup log describing what went wrong during testing and how it was fixed. There are also linked YouTube videos walking through the setup and results. This is aimed at people running large language models locally on their own hardware who want to understand whether speculative decoding techniques like DFlash are worth using, and how much faster they can expect generation to be at different context sizes. The full README is longer than what was shown.