optimalscale/lmflow

LMFlow is a Python toolkit for taking an existing large language model, the kind that powers chat assistants and text generators, and training it further on your own data so it behaves differently or knows about a specific domain. This process is called finetuning. Rather than building a model from scratch, which requires enormous computing resources, finetuning starts from a model that already understands language and adjusts it with a smaller set of examples. LMFlow is designed to make that process more accessible to researchers and developers. The toolkit supports a range of training approaches beyond basic finetuning. It includes methods for making models follow instructions better, for aligning model behavior with human preferences, for running models more efficiently in memory-constrained settings, and for speeding up text generation at inference time. Recent updates added support for a Hugging Face library called Accelerate, which handles running training across multiple GPUs or machines. On the model side, LMFlow works with many of the well-known open-source language models, including the Llama family, ChatGLM, Baichuan, Phi-3, and others. Users provide their training data in a format the toolkit specifies, configure a shell script with their chosen options, and run the training job. The README and documentation cover dataset formatting, conversation templates for different model families, and scripts for various training configurations. The project also released its own finetuned models under the name Robin, based on Llama, and published benchmarks comparing open-source chat models. A web demo is available at lmflow.com. The codebase is open source under the Apache 2.0 license, requires Python 3.9 or later, and can be installed via pip. Community support is available through Discord, Slack, and WeChat channels. The project received a Best Demo Paper award at the NAACL 2024 academic conference.