makan0713/onnx-web-worker-starter

This project is a small starter kit for running ONNX models, a common format for trained machine learning models, entirely inside a web browser without sending anything to a server. Instead of doing that inference work directly in the browser's main thread, where it can freeze up the interface while it runs, this starter moves the model loading and prediction work into a Web Worker, a separate background thread the browser provides, so the page stays responsive while a model is running. The author built this after running into problems where browser based inference felt fine on a desktop computer but became painful on mobile devices, and pulled the pattern out as reusable plumbing you can drop into your own project. It gives you a worker file that loads the ONNX Runtime library, a small wrapper class that lets your main code talk to that worker using promises, and an optional React hook for apps built with the React framework, including Next.js. You still need to bring your own trained model, the code to prepare its inputs, and your own interface design. One notable detail is that the worker file and the ONNX Runtime files need to live in your app's public folder with a fixed, predictable location, rather than being bundled the more typical way frameworks like Next.js usually handle files, because the way the worker loads its dependencies requires stable, unchanging file paths. To use it, you install the ONNX Runtime Web package, copy the worker and runtime files into your app's public folder, and place your own ONNX model file somewhere public as well. From there you call functions like preload, to warm up the model ahead of time, and run, to send input data in and get a prediction back out. It also includes a note that a worker keeps your interface responsive but does not make large models fast to run, and suggests shrinking image inputs before inference where possible. The project is written in JavaScript, was extracted from a real product called Lumli, and is released under the MIT license.