lisj575/omnidirector

OmniDirector is the code release for a research paper by teams at Kuaishou Technology, Tsinghua University, and Peking University. The system addresses a specific problem in AI video generation: taking camera movements from a multi-shot video and applying those same movements to animate a still image, without needing a training dataset that pairs the source and target footage together. The core idea is a "camera grid representation," which is the paper's proposed way of encoding camera motion from a sequence of different shots into a format that a model can learn from and apply. Alongside this, the system uses what the authors call a hierarchical prompt expansion agent, which combines multiple types of input signals, including text descriptions and visual cues, to guide the animation process. The practical use case is in video production contexts where you have a reference video whose camera work you want to reproduce: pans, zooms, tracking shots, and so on. OmniDirector is designed to apply those movements to a different source image, effectively letting users direct the camera of a new video by example rather than by manual keyframing. The README for this repository is sparse and contains mainly the paper abstract, author list, citation block, and a link to the project page and arXiv preprint. It does not include installation instructions, code usage examples, or model weights in the README itself. Readers looking for implementation details would need to consult the project page or the arXiv paper directly.