cdfan0627/longe2v

LongE2V is the official code release for a SIGGRAPH 2026 research paper about turning event camera data into ordinary video. An event camera is a special kind of sensor that only records changes in a scene instead of full frames, which makes it fast and good in tricky lighting, but the raw data it produces is sparse and hard to view directly. This project uses a video diffusion model, a type of AI that generates images by gradually refining noise into a picture, to convert those sparse event streams into stable, watchable video. The repository supports three related tasks: reconstruction, which rebuilds video from events alone, prediction, which forecasts future frames, and frame interpolation, which fills in smooth motion between two frames. All three tasks share one training and inference pipeline. Setup requires a Linux machine with an NVIDIA GPU and a specific CUDA version, since the code trains and runs on deep learning libraries like PyTorch. Datasets used in the paper (BS-ERGB, ECD, MVSEC, and HQF) are available preprocessed on Hugging Face, or you can generate the required voxel grids yourself from raw event files using the included scripts. Training is optional: the authors provide pretrained model weights so you can go straight to inference and see results without training anything. The README documents multiple stages for training, several inference scripts for each task, and postprocessing scripts that resize and correct outputs before comparing them to ground truth for scoring. This is a research codebase meant for reproducing academic results, not a polished consumer tool, so a background in machine learning and access to a capable GPU are expected.