datavorous/lensrt

LensRT is a diagnostic tool that helps you understand why machine learning models run slowly or fail to run efficiently on Qualcomm mobile processors. When you want to use a model on a Snapdragon phone or chip, the system tries to offload computation to specialized hardware (like a neural processor) to make it faster. This tool tells you exactly which parts of your model can be offloaded, which parts can't, and why. The tool works in two modes. The static mode analyzes your model file without running anything, it just reads the code and checks for common problems like unsupported operations or incompatible tensor shapes. The runtime mode goes deeper: it actually tries to compile your model using Qualcomm's tools and reports back what succeeded, what failed, and which original operations got grouped together into optimized blocks. Think of static mode as a checklist before you try something, and runtime mode as the real test drive. You'd use this if you're deploying AI models to Qualcomm-powered Android phones or embedded devices and want to squeeze out maximum performance. A concrete example: you built a language model and want it to run on Galaxy phones powered by Snapdragon processors. LensRT tells you which layers will run on the neural accelerator versus the CPU, which operations Qualcomm's SDK doesn't support yet, and sometimes even hints at whether it's a known issue or something in how the model was exported. The README includes a table of common problems like quantization mismatches or data type incompatibilities that block delegation. The tradeoff here is setup complexity. Running the full runtime diagnostics requires building Google's LiteRT framework from source (15, 30 minutes) and downloading Qualcomm's AI Runtime SDK, which needs a Qualcomm account. If you only need the quick static analysis, installation is simple. The tool is relatively new and niche, it's aimed at engineers specifically optimizing for Qualcomm hardware, not general deep learning work.