xiaomirobotics/xiaomi-robotics-u0

Xiaomi-Robotics-U0 is a large AI model from Xiaomi's robotics research team that is built to understand and generate text, images, and robot sensor style data all through one shared system. Rather than having separate models for each type of task, it treats everything, words and pictures and robot observations, as pieces of the same kind of information, which lets one model handle several different jobs. The model can do five things: turn a text description into an image, take an existing image plus an instruction and produce an edited or new image, generate multiple camera views of a scene from a text description of a task, convert an existing robot observation into a target view of a scene, and generate a short video showing a robot carrying out a task starting from an initial view. These are aimed at robotics researchers who need to simulate or generate training data for how a robot might see and interact with its environment. A companion technique called FlashAR speeds up the image generation process significantly, letting the model produce a high resolution image in about five seconds on a single high end graphics card, which the authors report is dozens of times faster than the model's original, unoptimized approach. Getting this running requires real technical setup: you need a capable GPU, need to create a specific Python environment depending on which inference approach you want to use, and need to download model weights that are hosted separately on Hugging Face or ModelScope. There is also an optional add-on for converting reference images into depth maps to support one of the five tasks. A Gradio based demo interface is included for trying the model interactively once everything is installed. The project is released under the Apache 2.0 license, and the code was published alongside an academic paper describing the research in detail.