vcruz305/llm-dequant

llm-dequant is a command line tool that converts AI language model files from a heavily compressed format called NVFP4 back into a standard, dense file format that most other tools can read. Some newly released models are only published in this compressed NVFP4 form, which locks them out of much of the surrounding ecosystem: tools that convert models to other formats cannot read it, fine-tuning frameworks expect uncompressed weights, and any kind of manual editing of a model's internal numbers needs those uncompressed weights too. The usual way to undo this compression loads the entire model into memory at once, which does not work for very large models with hundreds of billions of parameters. llm-dequant solves this by processing the model piece by piece rather than all at once, so memory use stays low and roughly constant no matter how large the model is. The tool converts to a bit-for-bit exact reproduction of what the reference decompression method would produce, and its output looks like a normal, standard model checkpoint, with the compression specific metadata removed and the tokenizer files copied over unchanged. It depends on only a small number of libraries and has no dependency on the specific compression library that created the file in the first place. The tool includes a verification command that samples parts of the actual checkpoint you plan to convert, runs the math forward and backward, and checks that the result matches byte for byte before you commit to a conversion that could take a long time and use a lot of disk space on a very large model. It also has a test suite that checks its work against hand computed examples and the original reference implementation, and it deliberately refuses to run rather than silently produce a wrong result if it encounters an unexpected file format, suspicious file paths, or invalid numeric values. Installation is a normal Python package install, and the typical flow is to inspect a checkpoint, verify that the math checks out, then run the conversion, after which the output can be fed into other standard tools such as scripts that build GGUF files for use with llama.cpp. The project only supports one specific compression variant, the one currently produced by the llm-compressor project, and it explicitly does not attempt to support other compressed formats. It is released under the Apache 2.0 license.