alayalab/alayaworld

AlayaWorld is a research project that generates playable, long running video worlds from a single starting image. You give it a first frame image, a camera path describing how the viewpoint should move, and a text prompt, and the model produces video chunk by chunk that follows that camera path while responding to the prompt. Each chunk is about 1.33 seconds of footage at 24 frames per second, and roughly 45 chunks add up to about one minute of generated video. The README describes four properties the model is built around. Interaction comes from a rendered 3D cache combined with camera control, plus the ability to switch prompts partway through generation to introduce new events. Consistency comes from two kinds of memory: a 3D cache that lets the model recall what a location looked like when revisited, and a compressed history of past frames for short term continuity. Stability is addressed by training on histories that include drift and by feeding accumulated errors back into training so mistakes do not pile up over a long rollout. Runtime performance comes from a distillation technique and short chunk sizes that aim to keep generation close to real time. Running the model requires a CUDA capable GPU and a fairly recent version of PyTorch. Setup involves downloading several separate sets of weights, including the project's own model file, a Gemma text encoder from Google that requires accepting a license first, and a depth estimation model called Depth Anything 3 that has to be cloned and installed as its own repository. A ready made example case is included so a new user can run one command and get a roughly one minute video out, and custom inputs are supported by providing a starting image, a camera trajectory file, and a text prompt. The code and weights are built on top of LTX-2 by Lightricks and are released under the LTX-2 Community License, which the README states is for academic research and non-commercial use only. Commercial use requires a separate license from LTX-2's owners. Only inference code and pretrained weights have been released so far, training code and full training data are marked as not yet available.