fudancvl/occlusionformer

OcclusionFormer is a research project from Fudan University that accompanies a paper accepted at the ICML 2026 machine learning conference. It tackles a specific problem in image generation: when you tell an AI model to draw a scene by giving it bounding boxes for each object, and those boxes overlap, current methods often blend the textures together or get the order wrong, so an object that should be behind ends up looking like it is in front. The authors propose handling the front to back order, which they call Z-order, as an explicit step in the model. The approach has three pieces according to the README: each object instance is generated separately, then composed using a method borrowed from volume rendering that decides how much each layer shows through, and finally a queried alignment step keeps each object in its correct spatial position. Alongside the model the team is releasing a dataset called SA-Z, which adds occlusion order and amodal annotations (information about parts of objects hidden behind other objects) to layout data. This repository is the inference and demo package. It contains the model code, a Streamlit web demo, a command line inference script, example layout JSON files, and a requirements file. The model weights and the SA-Z dataset are hosted on Hugging Face, and the paper itself is on arXiv. To run it, the README walks through creating a Python 3.11 conda environment, installing the requirements, downloading the checkpoint, and either starting the Streamlit demo or calling the CLI script with paths to a base FLUX model, the OcclusionFormer checkpoint, and a layout JSON. One open task remains: organizing the amodal annotations on Hugging Face.