tleyden/tauri2-local-llm

This repository is a prototype exploring how to run the Gemma 4 12B AI model, including its ability to understand audio, inside a desktop app built with Tauri, a framework for building lightweight apps using web technology paired with Rust. It is the third in a series of small prototype projects by the same developer, who is testing different ways to run AI models locally on a device rather than through a cloud service, for two apps they are building: a language learning tool and a markdown note taking assistant. The author's core requirements were getting Gemma 4 12B working with its built in audio understanding feature, running on macOS first, with support for other operating systems and models planned later. The README walks through five different technical approaches they considered for wiring the AI model into the Tauri app, listing the specific risks of each, such as whether a ready made plugin actually supports Gemma 4, whether a separate background process could be left running by accident, or whether a particular toolchain lacked audio support yet. After comparing these options, the developer settled on directly linking to a project called llama.cpp, a widely used tool for running AI models efficiently, through Rust's foreign function interface, a way for Rust code to call into code written in another language. They report this approach working well, and include a small benchmark showing how many tokens the model generates per second when processing a test audio clip on an Apple M2 Max laptop with 64 gigabytes of memory. This is explicitly early stage, exploratory work rather than a finished product, meant to document one developer's technical decision making process for others facing the same problem.