hiyouga/chatglm-efficient-tuning

ChatGLM-6B is an open-source AI language model developed by Tsinghua University, capable of conducting conversations in both Chinese and English. ChatGLM Efficient Tuning is a Python toolkit for adjusting, or fine-tuning, this model on your own data, so it learns to follow your specific instructions or domain knowledge rather than its general training. Fine-tuning a large language model from scratch is expensive because it requires updating all the model's parameters. This project supports several techniques that make the process much cheaper. The main method is LoRA, which adds small trainable weight matrices on top of the frozen model, updating only those instead of the full set of billions of parameters. The project also supports P-Tuning V2, a method that adjusts how the model reads its input context, and a simpler freeze approach that only updates the final layers. Full tuning of all parameters is available for cases where compute is not a constraint. A web browser interface is included, built with Gradio, which means you can run training, evaluation, and inference without writing command-line instructions. The project also supports Reinforcement Learning with Human Feedback (RLHF), a technique used to align model output with human preferences by training a reward model on human comparisons. A demo API server is included that matches the OpenAI chat format, so the fine-tuned model can be dropped into existing ChatGPT-based applications. The project includes a large collection of supported datasets in English and Chinese, covering general chat, math, code, and web content. Some datasets require a Hugging Face account and access approval before downloading. Custom datasets can be added by providing a JSON file in the expected format and updating a dataset registry file. The maintainer has since moved active development to a successor project called LLaMA-Factory, which covers a broader range of models. This repository is no longer being updated. Users who want to fine-tune ChatGLM2-6B or later models are directed to that newer project.