openhelix-team/robomemarena

RoboMemArena is a benchmark for testing how well robot manipulation systems handle memory, meaning tasks where the robot needs to remember something from earlier in a sequence to complete a later step correctly. The project comes from a research team and pairs with an academic paper, so its README reads more like a dataset and evaluation guide than a typical software project. At its core, the benchmark includes 26 separate manipulation tasks, meaning 26 distinct pick and place or object handling challenges a robot control system can be tested against. Alongside the tasks, the project provides demonstration data, which are recorded examples of a robot successfully performing each task, and BDDL files, which are formal descriptions of what each task requires the robot to accomplish. The dataset itself is hosted separately on Hugging Face and organized into four broad category folders, each containing individual task subfolders. Inside those subfolders are HDF5 files, a common format for storing large numerical data, holding camera images from two viewpoints, the robot's joint and gripper states, and the actions taken during each recorded demonstration. The repository also includes tooling to convert this HDF5 data into RLDS format, which is a standard format used by several robot learning pipelines, using a Python script and a specific set of package versions. Evaluation runs through a LIBERO compatible environment, a simulation setup common in robotics research, which this repo bundles as a local fork alongside code to connect with a runtime called OpenPI. There is also a smaller add-on folder called PrediMem S2 Training, which provides reusable pieces for training a Predictive Coding Head, a component used inside a Qwen3-VL style training pipeline. This add-on has its own separate README. Overall, this project is aimed at researchers building or evaluating robot control systems that need to reason over memory across a sequence of actions. It is a specialized benchmark, not a general purpose robotics or software library, and the README does not mention a software license.