federicobarrosgiuffrida/fluxinfer

FluxInfer is a command line tool that helps you get the best performance out of llama.cpp, a popular program for running AI language models on your own computer. Instead of guessing which settings will work well on your specific hardware, FluxInfer looks at your CPU, memory, and GPU, then automatically runs a small series of benchmark tests using a given model file to find the fastest working configuration. It saves that result as a reusable profile and can then launch the model for you using the settings it found. Importantly, FluxInfer does not build or modify llama.cpp itself. It only finds and runs the existing llama-bench, llama-cli, and llama-server programs that you have already built or installed separately, either through a folder you point it to, an environment variable, or your system's PATH. The tool reads real metadata directly from the model's GGUF file, such as how many layers it has, so its benchmark search is tailored to that specific model rather than using generic guesses. The project is explicitly labeled experimental. It has only been tested on one machine with two AI models and one graphics card so far, and the author is asking people with different hardware to share their results by opening an issue. Building it requires a C++20 compiler and CMake, with instructions given for both Windows and Linux, macOS is not yet supported. Most of its C++ dependencies are bundled directly in the project so a normal build does not need extra downloads. FluxInfer is released under the PolyForm Noncommercial License 1.0.0, which allows free use for personal, research, hobby, and nonprofit or educational purposes, but requires a separate paid license for any commercial use.