pointw/mofpo

This repository is the official research code for a paper called Mixture of Frames Policy, which is about teaching a robot with two arms to move around and manipulate objects. The core idea is that when the robot decides how to move, the code looks at that decision from several different points of view at once, such as relative to the robot's base, relative to its left hand, relative to its right hand, and relative to the path it is following. It then either lets a learned component pick the best view for each moment, or simply averages all the views together, and the paper shows this improves how well the robot performs compared to using only one point of view. The code here covers only the simulation experiments from the paper, tested on two existing robot simulation suites called BiGym and DexMimicGen. A separate related project, linked as a submodule, holds the real-world robot experiments described in an appendix of the paper. Setting this up requires a Linux machine with a GPU, a specific version of the MuJoCo physics simulator, and a conda-based Python environment with exact package versions pinned by the authors. Training data, meaning recorded example demonstrations of the robot completing tasks, can be downloaded from a public dataset the authors host on HuggingFace. Nine tasks are used across the two simulators, covering things like flipping a cup, sorting kitchenware, threading a needle-like object, and loading a dishwasher. Training and evaluating a model is done by running a shell script with a chosen task and method configuration, and every experiment setting reported in the paper, such as how many example demonstrations to use or how many training steps to run, is preset in the configuration files rather than left for the user to guess. This is not a beginner-friendly tool. It is research code meant for people already working in robotics or machine learning who want to reproduce or build on the paper's results, and it requires real robotics simulation experience and GPU hardware to run.