microsoft/onnxruntime

ONNX Runtime is Microsoft's open-source, cross-platform machine learning inference and training accelerator written in C++. ONNX (Open Neural Network Exchange) is a standard format for representing machine learning models, and ONNX Runtime is the engine that runs those models efficiently across different hardware and operating systems. For inference, running a trained model to make predictions, ONNX Runtime supports models from deep learning frameworks like PyTorch and TensorFlow/Keras, as well as classical machine learning libraries like scikit-learn, LightGBM, and XGBoost. It delivers faster performance by leveraging hardware accelerators where available and applying graph optimizations and transforms to the model. It is compatible with different hardware, drivers, and operating systems. For training, ONNX Runtime can accelerate model training time on multi-node NVIDIA GPU setups for transformer models, requiring only a one-line addition to existing PyTorch training scripts. The library is used to reduce inference costs and latency in production machine learning deployments. APIs are available for Python, C#, C++, Java, JavaScript (including web browsers and Node.js), and other languages. The project is MIT-licensed.