chiennv2000/orthrus

Orthrus is a technique that makes large language models, the AI systems behind tools like chatbots, produce answers much faster without changing what they actually say. Normally, an AI language model generates text one word at a time in a strict sequence, where each new word waits for the previous one to finish. That sequential process is the main speed bottleneck. Orthrus breaks that bottleneck by running two views of the same model at once: one view uses the standard sequential approach to stay accurate, and a second diffusion based view generates multiple tokens in parallel. The two views check against each other through an internal consensus mechanism, so the final output is guaranteed to match exactly what the original model would have produced on its own, meaning no quality is sacrificed for the speed gain. The practical result, according to the README's own benchmarks, is a speedup that reaches roughly 4 to 5 times on the released models and up to about 7.8 times on certain generation tasks, with no accuracy loss. It also needs no extra memory beyond the base model, because both views share the exact same internal cache. Only 16 percent of the model's parameters need to be fine-tuned to add this capability, while the core model stays frozen and untouched. Three ready-to-use models, built on the Qwen3 model family at 1.7B, 4B, and 8B parameter sizes, are published on Hugging Face along with example Python code for loading and running them, including a version you can try instantly in a hosted Colab notebook without installing anything locally. Researchers and developers building AI applications would use Orthrus when inference speed is a bottleneck and they cannot afford any drop in output quality. It requires a compatible GPU setup and the flash attention library to run efficiently.