robomotic/sesamerl

sesameRL is a Python-based reinforcement learning training environment for the Sesame robot, a four-legged robot with eight joints (described as 4-legged, 8-DOF, degrees of freedom). Reinforcement learning is a training technique where an AI agent learns to perform a task by receiving rewards for good behavior and penalties for bad behavior, gradually improving through repeated trial and error in a simulated environment. This project provides the simulation setup, training code, and configuration needed to teach the Sesame robot how to walk. The simulation is built on a tool called mjlab, and the learning algorithm used is PPO (Proximal Policy Optimization), a widely used approach for training locomotion policies. Every key setting, observations, rewards, curriculum stages, and training parameters, is concentrated in a single config.py file to make tuning easier. The trained policy is a small neural network with two hidden layers. During training it takes 36 inputs, including the robot's velocity estimates, orientation from an IMU sensor (inertial measurement unit, which tracks movement and tilt), joint positions and velocities, and a movement command. It outputs eight joint position targets, one per servo. After training, only the actor network is needed on the physical robot, the critic half is discarded. The repository also documents the sim-to-real gap: the physical robot uses an MPU6050 sensor for orientation and gyroscope data, MG90S servos for joints, and an ESP32S3 microcontroller to run the policy. Known limitations include servo timing latency and the fact that linear velocity must be estimated rather than measured directly on the hardware.