zju-real/sdar

SDAR is a research codebase implementing a method called Self-Distilled Agentic Reinforcement Learning. It addresses a challenge in training AI agents that need to take sequences of actions to complete tasks, such as navigating a simulated home environment or browsing an online store. Standard reinforcement learning (a technique where an AI learns by trial and reward) can be slow or unstable when applied to these multi-step agentic tasks. SDAR introduces a self-distillation step, where the agent periodically learns from its own past successful trajectories to stabilize and accelerate training. The code is evaluated on three benchmarks: ALFWorld (a text-based home navigation and task completion environment), WebShop (simulated online shopping requiring search and decision-making), and Search-QA (question answering that requires querying a document retrieval system). The paper reports substantial improvements over a standard reinforcement learning baseline on all three. The implementation is in Python and builds on the verl and verl-agent reinforcement learning frameworks. Training involves large language models run via vLLM (a high-throughput serving library), and the Search environment requires a separate local document retrieval server using FAISS (a library for fast similarity search over large datasets). The setup is complex, requiring multiple conda environments and GPU memory in the range needed for multi-billion-parameter language model training. This is a research artifact released alongside a paper on arXiv and is intended for AI researchers working on language model agent training.