real-stanford/iphumi

iPhUMI is a handheld data collection tool built for researchers who train robots to perform physical tasks by watching human demonstrations. The name is pronounced eye foo me, and it works by turning an iPhone into part of a gripper device you hold in your hand while showing a robot how to complete a task. The project builds on an earlier design called UMI, replacing that design's GoPro camera with an iPhone instead. This swap brings three main advantages, according to the project. First, an iPhone app makes it simple to label tasks, record demonstrations, and export the collected data, and existing UMI grippers can be converted to iPhUMI grippers by 3D printing two extra parts. Second, the iPhone's built in location tracking lets people record data in real world settings with very little setup, and multiple iPhones can be paired together so that two hands, or a head mounted phone, share the same coordinate system. Third, the app is open source, so people can adjust the data collection process to fit their own sensors or add entirely new gripper designs. The repository includes two main pieces: an iOS app for collecting the data, found in the ios_app folder, and a Python package called iphumi for processing that data afterward, found in python_package. The app also supports streaming data directly to a robot over USB or Ethernet so an iPhone mounted on the robot can be used to run trained policies live. A written usage guide walks through the setup steps. The project encourages contributors to share things like new phone mounts, alternative gripper designs, or additional features, and points to a separate data sharing initiative where people can contribute their own collected iPhUMI datasets for others to use. Several outside research projects already build on iPhUMI, including work on memory based policies, mobile manipulation, and force sensing. iPhUMI is released under the MIT license, and the project has 34 stars on GitHub.