tencent-hunyuan/unirl

UniRL is a software framework from Tencent's Hunyuan team for training AI models using a method called reinforcement learning. Reinforcement learning is a way of improving a model by having it try things, scoring how well it did, and then adjusting it to do better next time. UniRL applies this same training loop across many different kinds of AI models so that one system can handle them all. The models it supports are described as multimodal, meaning they work with more than one type of content. The list includes models that turn text into images, models that turn text or images into video, models that read both text and images and respond with text, and plain text language models. It also covers unified models that combine two different generation techniques. The README presents model support and algorithm support as two separate dimensions that can be mixed, so the framework covers many more combinations than the ready-made examples show. The project is organized in layers. You choose an entry point for the kind of model you want to train, load an example configuration file that describes the model, the algorithm, the rewards, and other settings, and the framework then runs the training loop for you. The shared underlying machinery handles spreading the work across many graphics processors, a common need because these models are large. A highlight the authors emphasize is two training algorithms their own team designed, called Flow-DPPO and DRPO, each with an accompanying research paper and a step-by-step tutorial. The framework also includes several well-known reference algorithms for comparison. To get started, you install the dependencies, run a check on your configuration, and then launch one of the provided example experiments with a single command, either on one machine or across several. The README lists a roadmap for adding more models and algorithms, explains how to contribute, and credits other open source projects the framework builds on. It is released under the Apache 2.0 License. This is a research and engineering tool aimed at people who train large AI models, not an end-user application.