paddlepaddle/plsc

PLSC is an open-source toolkit that helps you train AI models to recognize and classify things at massive scale, whether that's identifying faces among millions of people or sorting images into thousands of categories. It's built on top of PaddlePaddle, a deep learning framework, and is designed to handle classification problems where the number of possible categories is enormous. What makes it notable is its ability to scale. The project has demonstrated support for up to 92 million classes running on a single machine with 8 GPUs. It uses a technique called FP16 training (which uses lower-precision numbers) to speed things up while keeping accuracy intact, and supports a method called PartialFC that makes face recognition training more efficient when dealing with huge numbers of identities. It also supports distributed training across multiple machines. The toolkit includes a wide variety of pre-built models: face recognition approaches like ArcFace and CosFace, vision transformer architectures like ViT, Swin, DeiT, and CaiT, and self-supervised learning methods like MoCo, MAE, and CAE. Each model can reportedly be trained from scratch to match the accuracy reported in its original research paper. Who would use this? A startup building a face recognition product could use it to train models that distinguish between tens of millions of faces. A research team experimenting with vision transformers could use it to pre-train models on large image datasets and then fine-tune them for specific tasks like medical image classification or product recognition. Essentially, anyone who needs to train image classification models where the number of categories is too large for standard tools to handle efficiently. The README doesn't go into much detail about the specific workflows or how to prepare data, instead pointing to separate tutorials. Installation is straightforward, you can install it as a library via pip or clone the repository for local development.