paddlepaddle/graphnet

GraphNet is a benchmark dataset for researchers working on tensor compilers, the behind-the-scenes tools that make deep learning models run faster on hardware. It collects over 2,700 "computation graphs" (essentially blueprints of how a neural network processes data) extracted from real, state-of-the-art models across natural language processing, computer vision, and other tasks. The goal is to give compiler researchers a shared, standardized set of test cases so they can fairly compare how well different compilers optimize performance. When a deep learning model runs, a tensor compiler translates its computation graph into efficient machine code. Different compilers, like PyTorch's TorchInductor or PaddlePaddle's CINN, take different approaches, and until now there hasn't been a large, common dataset to evaluate them against. GraphNet fills that gap by providing models in a standardized format with rich metadata, so researchers can run a compiler across all 2,700+ graphs, measure speedup against a baseline, and see where it shines or struggles. The project also introduces two custom metrics: one that scores raw speedup, and another that factors in compilation or runtime errors, giving a more realistic picture of a compiler's practical reliability. The primary audience is tensor compiler developers and researchers, particularly those exploring "AI for Compilers", using machine learning to automatically improve compiler optimization strategies. A hardware company testing whether their new chip works well with existing compilers could use this dataset as a regression test suite. A compiler team deciding whether a new optimization actually helps across diverse models, or a researcher training a model to predict good optimization strategies, would also find it valuable. The roadmap includes expanding to 10,000+ graphs, adding multi-GPU scenarios, and enabling subgraph-level analysis, pointing toward increasingly granular and large-scale evaluation.