oliverleexz/serl

SERL is an AI research codebase implementing a technique called "Selective Hindsight Distillation" for training AI language model agents that take actions over many steps, for example, navigating a virtual household or shopping online. The core research problem: training these agents with reinforcement learning (learning from trial and error) is difficult because rewards are sparse, the agent often only learns whether it succeeded at the very end of a long task, not after individual steps. SERL addresses this by having a second AI model (the "teacher") look at what happened in hindsight and provide richer feedback signals for each action the student agent took. The selective aspect is important: SERL applies this teacher feedback only to the action tokens (the actual decisions the agent makes), not to the chain-of-thought reasoning tokens (the agent's internal thinking). This way, the feedback guides what the agent does without overwriting how it reasons. The system is tested on two standard AI agent benchmarks: ALFWorld (a text-based household navigation environment where an agent must find and manipulate objects to complete tasks) and WebShop (a simulated online shopping environment). This is a research implementation intended for machine learning practitioners. It requires significant computational infrastructure and specific deep learning library versions to run. The full README is longer than what was provided.