ammarkov/sam3dbody-cpp

SAM3DBody-cpp is a C++ program that takes video from a single ordinary camera and produces a three-dimensional model of every person's body and hands visible in each frame, in real time. It does not require depth cameras, multiple camera setups, special sensors, or any Python installation to run. The entire inference pipeline runs through compiled C++ code. What it produces is a standard motion-capture file format called BVH for each detected person in the video. These files record how every joint in the body, including the hands, moves from frame to frame. You can open these files directly in animation software like Blender, and a bundled Blender plugin is included to drive a character rig from the results. Each person in a multi-person scene gets their own BVH file, and the software keeps track of which person is which across frames so the identities do not swap. The system works by running a sequence of neural network models. A detection model finds people in the image. A large visual understanding model then analyzes each person's crop and produces a compact description of their pose. A final set of lightweight models decode that description into the 519 specific numbers that describe a full-body skeleton with hand poses and even basic facial expressions. A linear blend skinning step then converts those numbers into 18,439 surface points forming the body mesh, plus 70 labeled joint positions. Running the GPU-accelerated version requires a CUDA-capable graphics card and about 5 gigabytes of downloaded model files from HuggingFace. A CPU-only version exists but processes a single frame in 5 to 15 seconds depending on the computer, which is not practical for video. Building from source requires CMake and a C++ compiler, ONNX Runtime and ggml handle the model execution. Python frontends are included for users who want to call the compiled library from Python scripts without rewriting any C++ code. A CSV exporter is also available for the 70 joint positions if you need the data in spreadsheet form. The full README is longer than what was shown.