sebastian-stapf/world-model-self-distillation

This repository holds the official project materials for a research paper titled "World Model Self-Distillation: Training World Models to Solve General Tasks." The actual code has not been released yet, what is here is the project abstract, links to data, and links to pretrained model weights. The research is about teaching AI systems to watch a scene and then figure out how to complete a task in that scene, without needing a library of pre-recorded demonstrations showing how to do each task. The idea behind "world models" in this context is an AI that has learned, from video data, how actions tend to play out over time in the physical world. The method, called WMSD, starts with AI models that were originally trained to generate videos. It adapts those models in two stages. First, the system generates written instructions and step-by-step solution descriptions directly from scene images. Second, it trains one version of the model (called the Executor) to follow short instructions, guided by a more detailed version (called the Demonstrator) that has access to the full solution. A separate AI reviews the Executor outputs and provides feedback to keep improving it, while the original video model acts as a stabilizing reference so the adapted model does not drift too far. The paper tests this approach on a benchmark called WorldTasksBench, using two base video models. Results show improvements in how often the model completes tasks correctly and how physically realistic the outputs look, compared to the starting models. The repository is sparse by design. No training code is available yet. The authors link out to a dataset and pretrained weights on Hugging Face, along with a project page with more detail.