xinyan-cxy/opencof

OpenCoF is the official repository for a research paper titled OpenCoF: Learning to Reason Through Video Generation. The paper's idea is that video generation models can reason in a different way than text based chain of thought models do, by working through a sequence of connected video frames, which the authors call Chain of Frame reasoning. They argue that existing video generation models are trained on general video data and are not specifically set up to support this kind of frame by frame reasoning. To study this, the authors built OpenCoF, a framework centered on a new dataset called OpenCoF 17K, which contains 17,312 videos spanning 11 different task types. The dataset was put together using four different methods: instance based rendering, expert guided rendering, procedural scene generation, and repurposing existing external videos. They used this dataset to fine tune an existing video model, called Wan2.2-I2V-A14B, into a new version called Wan-CoF, which they report performs better on four external video reasoning benchmarks. They also tested two additional designs, called Visual Reasoning Tokens and Textual Reasoning Tokens, which aim to capture lower level visual details and higher level meaning separately, producing two further model versions with additional gains. As of this README, the repository contains only the paper description and figures. The authors state clearly that the code, the OpenCoF-17K dataset, and the Wan-CoF model checkpoints are still going through internal company review and are not yet available, listing all three as coming soon. There is a link to a project page and to the paper on arXiv, along with a citation entry for anyone referencing the work, but no license is stated in the README and there is nothing here yet to install or run.