charlesq34/pointnet2

PointNet++ is a research project from Stanford University focused on teaching computers to understand 3D shapes. The input it works with is called a point cloud: a large set of (x, y, z) coordinate measurements, the kind of data produced by a 3D sensor or LiDAR scanner. Rather than converting that data into images or meshes first, this system works directly with the raw coordinate points. The original PointNet (version 1) processed each point in isolation or looked at the entire set as one global summary, missing the in-between spatial structure. PointNet++ improves on that by working hierarchically, meaning it first groups nearby points together, extracts local patterns from those groups, then combines those patterns into bigger-picture patterns at larger scales. The approach is similar to how image recognition networks zoom out through layers, except here the input is 3D coordinates rather than pixels. The system also handles the fact that real-world 3D scans are not evenly spread out: some areas have dense clusters of points and others are sparse, so special layers are included to handle those varying densities. The code in this repository lets you train and evaluate two main tasks. The first is shape classification, where the model learns to identify what kind of object a point cloud represents, tested on a dataset called ModelNet40 with 40 categories of 3D models. The second is part segmentation, where the model labels which part of an object each point belongs to, such as distinguishing the seat, back, and legs of a chair, using a dataset called ShapeNet. A third task, semantic scene parsing of indoor environments using the ScanNet dataset, is also included with its own training script. The project was originally built on TensorFlow 1.2 with Python 2.7, which are older versions. Custom TensorFlow operators need to be compiled from source before use, a step that requires some familiarity with build tools. GPU access is strongly recommended. The code is released under the MIT license. This work was originally published at the NeurIPS 2017 conference.