facebookresearch/pixio

Pixio is a vision model designed to understand images at a detailed, pixel-by-pixel level. Unlike general-purpose image models that learn broad concepts, Pixio is specifically trained to excel at tasks requiring precise spatial understanding, like estimating depth from a single photograph, identifying what's in each part of an image (segmentation), or reconstructing 3D scenes from photos. The key insight is that by teaching the model to reconstruct pixels during training, it naturally learns fine-grained visual features that transfer well to these downstream tasks. The model works by starting with a proven training approach called masked autoencoding. Think of it like a jigsaw puzzle: during training, parts of an image are hidden, and the model learns to predict what's missing. Pixio refines this idea with three specific improvements, a more sophisticated decoder network, larger masked regions, and multiple "class tokens" (special learned tokens that help aggregate information). These tweaks make the model much better at capturing spatial details. The researchers also trained it on a much larger, carefully curated dataset (MetaCLIP-2B) rather than the standard ImageNet, which further boosts performance. The numbers speak clearly: Pixio significantly outperforms competing models on real-world benchmarks. On depth estimation, it achieves 95.5% accuracy on the NYUv2 dataset compared to 93.2% for the previous state-of-the-art. On semantic segmentation and 3D reconstruction tasks, it shows similarly large improvements. What's remarkable is that Pixio achieves this with a model size (631M parameters) comparable to older approaches but much smaller than some competitors. Researchers and engineers building computer vision systems would use Pixio when they need a strong foundation model for tasks involving spatial precision. The repository includes pre-trained models in several sizes (86M to 5.4B parameters), code for fine-tuning on custom tasks, and evaluation scripts for standard benchmarks. You can use it directly via a simple API or integrate it with standard machine learning frameworks like Hugging Face Transformers, making it practical for both research and production applications.