zju-omniai/vla-corrector

vla-corrector is the research code accompanying a paper about improving robots that use Vision-Language-Action, or VLA, policies to decide what physical actions to take based on camera images and language instructions. These policies often plan several future actions at once, called a chunk, and then execute them in sequence without checking back in until the whole chunk finishes. That saves time but creates a blind spot: if something in the real world shifts partway through, like a part slipping or the robot's pose drifting, the robot keeps following its stale plan anyway. VLA-Corrector adds a small extra component on top of an existing, unmodified VLA policy rather than retraining the whole thing. A piece called the Latent-space Vision Monitor watches whether what the robot's cameras actually see during execution matches what was expected, using an internal, compressed representation of the visual scene rather than raw pixels. When it detects a persistent mismatch, it cuts off the remaining queued actions and triggers a single corrective replanning step, called Online Gradient Guidance, before letting the robot continue. The only part that gets trained is this lightweight corrector, reported at roughly forty million parameters, not the original policy. The paper reports test results across several robot manipulation benchmarks and one real robot arm, showing consistent improvement in task success rate when the corrector is added on top of existing policies such as PI0.5, SmolVLA, and X-VLA, with gains ranging from about four to eighteen percentage points depending on the setup. The repository does not ship any datasets, pretrained weights, or trained checkpoints. Users are expected to bring their own base VLA policy checkpoint and dataset, referencing specific Hugging Face model repositories named in the code, then run separate scripts to extract latent features, train the corrector, and evaluate the combined system. Setup uses a conda environment built from an included environment file, and the project is built on top of the open source LeRobot framework. The work comes from researchers at Zhejiang University and Alibaba DAMO Academy, with a project page, paper, and code linked from the README.