armanbilge/onnx-scala

ONNX-Scala lets you run pre-trained machine learning models from inside a Scala application. Instead of writing a separate Python script or service to do inference, you load a model file directly in your Scala code, feed it data, and get predictions back, all without leaving your JVM or JavaScript environment. At its core, the project wraps ONNX Runtime, Microsoft's engine for running trained models. You point it at a model file (like an image classifier), pass in your input data as a typed tensor (think of a tensor as a multi-dimensional array with labels like "Batch," "Channel," "Height," and "Width"), and call a method to run the model. The output comes back as another tensor with its own shape and labels. The project also offers a lower-level API for building models piece by piece, but the simplest path is loading a whole model file and letting the runtime optimize it. The main audience is Scala developers who want to use existing ONNX models without bridging to Python. For example, if you have a SqueezeNet model that classifies images into ImageNet categories, you can load the model, pass in an image array, and get back the predicted class index, all in Scala. The README notes that performance is within 3% of calling ONNX Runtime from Python directly, with the small gap coming from moving data between the JVM and native memory. One notable aspect is the type system. The project uses Scala 3 features to encode tensor shapes and axis labels at the compile-time level, meaning the compiler can catch shape mismatches before your code runs. The full-model API is more loosely typed on the input side since models are loaded from disk at runtime, but outputs are still typed. It also runs across multiple Scala platforms: the JVM, Scala.js (JavaScript), and Scala Native, so the same model-loading code can work in a browser or on a server. Training support is not yet available, the project currently focuses on inference (running predictions with existing models).