nvlabs/isaaclabeureka

Isaac Lab Eureka uses large language models (like GPT-4) to automatically design and tune reward functions for training robots. If you're building a robot and want it to learn a new skill, say, balancing a pole on a cart, you normally have to manually write complex mathematical rules telling the robot what "good" behavior looks like. This project removes that manual work by letting an AI figure out the rules for you. The tool works inside Isaac Lab, NVIDIA's robotics simulation platform. You give it a task (like "balance this cartpole") and an OpenAI API key. The system then loops through a process: the language model writes a candidate reward function, a simulated robot tries to learn the task using that function, and the system measures how well the robot performed. That performance feedback gets sent back to the language model, which refines its approach and tries again. Over multiple iterations, the AI homes in on reward functions that produce effective robot behavior. This is useful for robotics researchers and engineers who work with reinforcement learning but spend significant time hand-tuning reward functions, a notoriously tedious and finicky part of the process. For example, if you're training a simulated robot arm to grasp objects, you'd normally write careful rules balancing "reach the object" against "don't move too jerkily." Eureka automates that search and, according to the original research it implements, can match or exceed human-level quality on these designs. There are some limitations worth noting. It only supports tasks built in a specific Isaac Lab format, and it works with two particular reinforcement learning libraries (RSL RL and RL-Games). The language model sometimes generates buggy code, in which case that iteration simply gets skipped. On Windows, you can't run multiple training attempts in parallel. You also need an OpenAI or Azure OpenAI API key, since the language model does the heavy lifting. The project is open source under an MIT license and runs on both Linux and Windows.