general-instinct/instinctrazor

InstinctRazor is a research project that takes a very large AI language model and compresses it so it can run on a single high-end GPU instead of requiring four of them. The model in question, Qwen3.5-122B, is a 122 billion parameter model that normally requires about 245 gigabytes of storage. This project shrinks it to roughly 47 to 48 gigabytes by reducing the numerical precision of the model's internal values, a process called quantization. The core finding documented here is that this particular type of model architecture, called a Mixture of Experts, behaves unusually well under aggressive compression. In a standard AI model, compressing to low precision tends to meaningfully hurt quality. In this architecture, only a small fraction of the model's internal components are active at any one time when processing a given piece of text, which means the compression causes less real-world quality loss. The project's benchmarks show that the compressed model scores higher than a competing model of similar size on most standard knowledge and reasoning tests, while only losing ground on tasks that require generating very long outputs before arriving at an answer. The repository includes Python scripts for running the compression pipeline yourself, for evaluating the resulting model on standardized benchmarks, and for optionally fine-tuning the model to recover any lost performance on specific tasks. A ready-to-use compressed version of the model is also published separately on Hugging Face for people who want to download and run it without going through the compression process themselves. The project is structured so that researchers can verify the results on smaller hardware first using a lightweight test that runs the same code path on a smaller model. Full reproduction of the headline result requires four large GPUs. The README is detailed about what each benchmark measures, where gaps remain, and what the known limitations of the evaluation setup are.