tloen/alpaca-lora

Alpaca-LoRA is a toolkit for fine-tuning the LLaMA language model on consumer hardware, meaning a regular gaming GPU rather than expensive data-center machines. The core problem it addresses: training large AI language models normally requires hundreds of thousands of dollars in compute. This project uses a technique called LoRA (Low-Rank Adaptation), which adds a small set of trainable "adapter" weights on top of an existing frozen model, dramatically reducing memory and compute requirements. In practical terms, you can use this project to train a ChatGPT-style instruction-following AI model (one that responds to commands like "write me a poem" or "explain this concept") in just a few hours on a single high-end consumer GPU like an RTX 4090. The result is a model of similar quality to the Stanford Alpaca model, which itself was designed to approximate text-davinci-003. Once trained, the model can even run on a Raspberry Pi for research purposes. The repo provides scripts to fine-tune LLaMA models (7B, 13B, 30B, and 65B parameter sizes), generate responses through a Gradio web interface, and export the merged weights for use with other tools like llama.cpp. It uses Hugging Face's PEFT library and bitsandbytes for efficient training. Pre-trained LoRA adapter weights are also available on Hugging Face for those who just want to run the model without training.