nvlabs/instant-ngp

Instant Neural Graphics Primitives, or instant-ngp, is a research project from NVIDIA that turns photos or 3D data into interactive visual scenes you can fly around, and does it very quickly. The headline trick is training a NeRF, which stands for neural radiance field, of a fox from a small set of photos in under 5 seconds rather than the hours it traditionally took. Beyond NeRFs, the same code handles three other neural graphics primitives: signed distance functions (a way of describing 3D shapes), neural images (storing a single image inside a small neural network), and neural volumes for things like clouds and smoke. The way it works is that all four primitives are represented by a small neural network, a multilayer perceptron, combined with a multiresolution hash input encoding, the trick described in the team's SIGGRAPH 2022 paper. The network and encoding run on top of the project's tiny-cuda-nn library, which is what makes training fast enough to feel interactive. The application ships as a GUI where you drag a dataset or 3D file into the window and watch it train and render live, it also offers a camera path editor for making videos, snapshot save and load, a VR mode, and conversions from NeRF or SDF to a regular polygon mesh. You would use this to capture a real-world scene from a phone-camera video and explore it in 3D, prototype a NeRF-based effect, or experiment with these compression-style neural representations. Pre-built downloads are provided for Windows, Linux users and anyone wanting the Python bindings build from source. The code is written in CUDA for NVIDIA GPUs. The full README is longer than what was provided.