facebookresearch/sapiens2

Sapiens2 is a family of AI vision models from Facebook Research, specialized in understanding the human body in images and video. These are "vision transformers", a type of AI model that processes images by breaking them into small patches and learning relationships between those patches. The models were pre-trained on one billion human images, making them highly capable at recognizing body-related visual information. The models can perform five distinct tasks: pose estimation (detecting where body joints like elbows and knees are positioned), body-part segmentation (identifying which pixels in an image belong to which part of the body), surface normal estimation (calculating the 3D surface orientation of the body, useful for lighting and 3D effects), pointmap estimation (estimating 3D coordinates for each point on the body), and human matting (cleanly separating a person from their background at a pixel level). The models come in multiple sizes ranging from 0.1 billion to 5 billion parameters, more parameters generally means higher accuracy but more computational cost. They process images at 1,024 by 768 pixel resolution by default, with a special 4K version for very high-resolution work. Pre-trained model weights are available for download from HuggingFace. You would use Sapiens2 if you are building applications that need to understand human bodies in images or video, such as avatar animation, virtual try-on, augmented reality effects, or any creative tool that needs to understand body pose or separate people from backgrounds. It is written in Python and requires PyTorch.