nvlabs/eg3d

EG3D is an NVIDIA research project that generates realistic 3D objects, like human faces, cat heads, and cars, using only ordinary 2D photos for training. Instead of just producing flat images, it creates genuine 3D shapes you can rotate and view from any angle, all in real time. You can use pre-trained models to generate videos, images, and 3D shape files, or train new models on your own collections of photographs. Under the hood, the system uses a type of AI called a generative adversarial network (GAN), essentially two neural networks competing, where one creates images and the other judges them until the outputs look real. The key innovation is splitting the work into two parts: a 2D image generator (borrowing from NVIDIA's existing StyleGAN2 technology) handles the heavy lifting of producing detailed features, while a separate rendering step projects those features into 3D space from arbitrary camera angles. This separation lets it produce high-resolution images and accurate 3D geometry without the extreme computational cost of earlier approaches. Researchers and developers working on 3D content generation would use this to create synthetic 3D faces, animals, or vehicles without needing actual 3D scans. For example, you could generate a realistic human face, export it as a 3D shape file, and load it into a visualization tool to inspect the geometry from all sides. The project includes an interactive visualizer for exploring trained models, and you can generate interpolated videos that smoothly transition between different generated objects. The project is demanding to run. Training requires 1 to 8 high-end NVIDIA GPUs and specific CUDA software. It's also explicitly a research reference implementation, NVIDIA states they do not accept outside code contributions, so it's a snapshot of a published paper rather than an actively maintained product. The license section is commented out in the README, and NVIDIA directs commercial inquiries to a separate licensing page.