microsoft/onnxruntime

📈 Trending★ 20,539C++Audience · developerComplexity · 4/5ActiveLicenseSetup · moderate

Mindmap

mindmap
  root((ONNX Runtime))
    What it does
      Runs trained models
      Accelerates training
      Optimizes performance
    Supported models
      PyTorch
      TensorFlow
      scikit-learn
      XGBoost
    Hardware support
      NVIDIA GPUs
      Multi-platform
      Hardware accelerators
    Use cases
      Production inference
      Reduce latency
      Lower costs
    APIs available
      Python
      C++
      JavaScript
      Java

mindmap root((ONNX Runtime)) What it does Runs trained models Accelerates training Optimizes performance Supported models PyTorch TensorFlow scikit-learn XGBoost Hardware support NVIDIA GPUs Multi-platform Hardware accelerators Use cases Production inference Reduce latency Lower costs APIs available Python C++ JavaScript Java

Things people build with this

USE CASE 1

Deploy PyTorch or TensorFlow models to production with faster inference and lower latency.

USE CASE 2

Run machine learning models on edge devices or servers with limited resources.

USE CASE 3

Speed up transformer model training on multi-GPU clusters with a one-line code change.

USE CASE 4

Convert and optimize models from scikit-learn, XGBoost, or LightGBM for efficient serving.

Tech stack

C++PythonCUDAPyTorchTensorFlowONNX

Getting it running

Difficulty · moderate Time to first run · 30min

Requires C++ build toolchain and CUDA toolkit if GPU acceleration is desired; Python bindings available but compilation from source may be needed.

Use freely for any purpose, including commercial use, as long as you keep the copyright notice and license text.

In plain English

ONNX Runtime is Microsoft's open-source, cross-platform machine learning inference and training accelerator written in C++. ONNX (Open Neural Network Exchange) is a standard format for representing machine learning models, and ONNX Runtime is the engine that runs those models efficiently across different hardware and operating systems. For inference, running a trained model to make predictions, ONNX Runtime supports models from deep learning frameworks like PyTorch and TensorFlow/Keras, as well as classical machine learning libraries like scikit-learn, LightGBM, and XGBoost. It delivers faster performance by leveraging hardware accelerators where available and applying graph optimizations and transforms to the model. It is compatible with different hardware, drivers, and operating systems. For training, ONNX Runtime can accelerate model training time on multi-node NVIDIA GPU setups for transformer models, requiring only a one-line addition to existing PyTorch training scripts. The library is used to reduce inference costs and latency in production machine learning deployments. APIs are available for Python, C#, C++, Java, JavaScript (including web browsers and Node.js), and other languages. The project is MIT-licensed.

Copy-paste prompts

Prompt 1

Show me how to load a PyTorch model with ONNX Runtime and run inference in Python.

Prompt 2

How do I convert a TensorFlow model to ONNX format and optimize it with ONNX Runtime?

Prompt 3

Add ONNX Runtime training acceleration to my existing PyTorch distributed training script.

Prompt 4

What hardware accelerators does ONNX Runtime support, and how do I enable them?

Prompt 5

How do I deploy an ONNX model to a web browser using ONNX Runtime JavaScript?

Open on GitHub → Explain another repo

Generated 2026-05-18 · Model: sonnet-4-6 · Verify against the repo before relying on details.