open-x-humanoid/robo-valuerl

Robo-ValueRL is a research framework for training robots to perform physical tasks, combining offline learning from recorded demonstrations with online learning from real world practice. Its main idea is to train a model that watches a history of what a robot has seen and done and predicts how far along the robot is in completing its task. This prediction helps resolve confusion that comes from partial camera views or objects being blocked from view. That progress prediction is then turned into quality labels for individual actions, which are used to train a policy, meaning the actual decision making model that controls the robot. This policy is first pretrained on a large offline dataset of recorded demonstrations. Once pretrained, the policy is deployed onto a real robot, where a human can step in during test runs to correct or intervene when needed, and this real world data is used to further adapt and improve the policy through what the README calls online adaptation. The project is built around a dual arm humanoid robot with three cameras, one on the head and one on each wrist, and its deployment setup uses two machines: one running the AI model as a server, and a separate robot host that streams camera and sensor data to it and executes the actions it returns. Training the models requires substantial hardware, with the README specifying eight A100 GPUs across possibly multiple machines for training, while running the trained policy on a real robot needs only a single RTX 4090. Setup starts with a provided install script that creates a Python environment and installs the project's dependencies, including two other open source robotics projects it builds on. Users then download a released dataset from Hugging Face, formatted using the LeRobot standard, and there is also a script provided to convert your own raw robot recordings into that same format if you want to use your own data. Model checkpoints and the dataset are also published for public download. This is specialized robotics research software intended for people with real robot hardware and serious GPU infrastructure, not something meant to be tried casually.