dcdmllm/visualthink-vla

VisualThink-VLA is a research project for making AI-controlled robots act more accurately and with lower delay. It is tied to an academic paper and the code was made public in May 2026. The core idea is about how robots interpret camera images when deciding what physical action to take next. Most AI robot systems (called vision-language-action policies, or VLAs) feed raw images along with text instructions into a large model and ask it to decide on an action. VisualThink-VLA takes a different approach: instead of passing the full image, it first extracts compact pieces of visual evidence and only passes what is relevant for the current task step. The four types of evidence it can extract are bounding boxes around objects, edges and contours, motion differences between frames, and spatial relationship information derived from the text instruction. The system has a router that decides which of these four evidence types are needed for a given moment in a manipulation task, for example picking up a bowl versus placing it on a surface. The underlying base robot model is kept frozen, meaning no retraining is needed. Only the small routing and adapter modules are trained. This keeps training costs down and leaves the base policy untouched. The codebase includes scripts for extracting visual evidence from robot image sequences, training the router and adapters, building an auditable training dataset called VisualEvidence-Set, and running evaluations including a faithfulness audit and a success-versus-latency tradeoff plot. Installation requires Python 3.10 and a small set of packages, with optional dependencies for specific robot simulators and perception models. This is academic research code, not a production tool. It targets robotics researchers familiar with AI-based robot control systems.