wafer-bob/hallo4d

Hallo4D is a research project from a group of authors describing a method for fixing errors in AI generated 3D and 4D content, where 4D means a 3D scene that also changes over time, like a short animated clip. The README frames it as a companion to arXiv paper 2607.12752 and links to a project page rather than shipping much runnable code at this point. The idea behind Hallo4D is to catch and correct spatial and temporal inconsistencies, meaning things that look wrong from a different angle or that flicker and drift as an animation plays, in content produced by other 3D or 4D generation systems. It works by using large multi modal models, AI systems that can reason about both images and text together, to look at renderings from multiple viewpoints and multiple frames and flag places where the generated content contradicts itself. Several of these models vote together to decide what looks inconsistent, and that agreement is used to guide a correction process. Importantly, the README states this happens without retraining the original generation model or changing its architecture, so it can be added on top of an existing pipeline. The README also describes a temporal consistency step that picks out the most important frames to fix, based on motion, in order to reduce jitter and flickering identity between frames, and a separate step aimed at keeping exposure and brightness stable across viewpoints that are not facing the camera directly. Sample animated comparisons in the README show a baseline result next to the same scene with Hallo4D applied, described as tuning free. Hallo4D is presented as a follow up to an earlier project called Hallo3D, which focused only on making 3D generation consistent across multiple viewpoints and was published at NeurIPS 2024. The current repository is released under the MIT license.