code-byte404/microant

MicroAnt is a tool for measuring how much a small AI language model loses in capability when it is compressed through a process called quantization. Quantization makes a model smaller and faster to run by reducing the precision of its numbers, but it can also make the model worse at reasoning or following instructions. Many people repeat rules of thumb about how much compression is safe, but this project is built to actually test that question in a way anyone can repeat and check. The tool runs the same set of test tasks, called a capability pack, against a chosen model at several different compression levels, across two popular formats for running AI models locally: GGUF, used by the llama.cpp project, and MLX, used on Apple Silicon computers. It records where each compression level starts to fail at the tasks, and reports a confidence range around each score rather than a single misleading number. The tool intentionally does not build a leaderboard or focus on speed, it treats speed as just one extra cost figure next to the capability results. The workflow has four main steps: profiling a model at every compression tier, prescribing the lightest amount of extra help, called scaffolding, needed to bring a compressed model's answers back up to an acceptable quality bar, confirming that this fix actually holds on a separate set of held out test cases, and comparing results across different model sizes within the same family. According to the README, the project is currently proven end to end on one model family, and some reports for other models are temporarily removed while data is re downloaded. Installing MicroAnt requires Python and, to actually run models rather than just read past reports, an Apple Silicon Mac. It includes an automated test suite that checks the scoring and reporting logic without needing a GPU or any downloaded model weights, plus a small local web interface built with Streamlit for running comparisons without typing commands.