irvlutd/uhas_sim

This repository is the official code release for a robotics research paper called Cross-Embodiment Robot Manipulation via a Unified Hand Action Space, or UHAS. The core idea is that different robotic hands, like a four fingered hand or a five fingered hand, normally each need their own separately trained control policy because their joints and shapes are different. UHAS tries to solve that by representing any hand's motion as a deformation of a shared, imaginary sphere rather than as raw joint angles. A lightweight algorithm called Cascade Inverse Kinematics then translates those sphere deformations into the specific joint commands each real hand needs, and it can do this fast enough for real time control.\n\nBecause every hand's actions are described the same way, through this shared sphere representation, a single reinforcement learning policy can be trained once and then used to control several different robotic hands, including the LEAP Hand, Allegro Hand, Shadow Hand, and a simulated human hand model called MANO. The repository even includes a pre-trained policy that was trained jointly across all four of these hands, and it can reportedly be applied directly to a brand new hand it has never seen before, without retraining. The project provides everything needed to reproduce this: the pipeline that automatically builds the sphere representation from a hand's URDF description file, full simulation environments built on NVIDIA Isaac Lab and Isaac Sim for training and testing cube reorientation tasks, and scripts to train new policies or evaluate the pre-trained ones. There is also a guide explaining how to add support for an entirely new dexterous hand design. Running this project requires substantial setup: NVIDIA Isaac Sim 4.5.0, Isaac Lab 2.2.1, Python 3.10 or newer, a Conda environment, and a GPU with at least 16GB of video memory for running more than 1000 parallel training environments. Once installed, training and evaluating policies is done through command line scripts. The code is released under the Apache License 2.0, and the authors ask that anyone using the code or the UHAS idea cite their paper.