hiyouga/llamafactory

LlamaFactory is a Python toolkit for fine-tuning large language models (LLMs) and vision-language models (VLMs). Fine-tuning means taking a pre-trained AI model that has already learned from massive amounts of text or data and further training it on your specific dataset so it becomes specialized for your use case, for example making a general model become an expert at customer support conversations or medical question answering. LlamaFactory makes this process easier by providing a unified interface that supports over 100 different models with minimal or no coding required. The toolkit supports a range of training approaches beyond basic fine-tuning, including LoRA and QLoRA (which are parameter-efficient techniques that only update a small fraction of the model's weights to save memory and compute), reward modeling, and reinforcement learning from human feedback methods like PPO and DPO. It handles models in quantized 2 to 8-bit formats, which allows large models to be fine-tuned on consumer-grade GPUs with less memory. A web-based graphical interface called LLaMA Board, built with Gradio, lets users configure and launch training runs without writing any code. The command-line interface serves more advanced users. After training, models can be deployed with a vLLM-powered API that follows the OpenAI API format, making integration straightforward. You would use LlamaFactory if you are a researcher or developer who wants to customize a pre-trained LLM for a specific task, dataset, or domain without building all the training infrastructure from scratch. It is also suitable for cloud-based training via Google Colab or similar services for people who do not have local GPU hardware. The tech stack is Python, PyTorch, and Hugging Face libraries, with Docker support for reproducible environments. It was published as a paper at ACL 2024.