projectargus-cc/libargus.cc

libargus is a low-level software library that lets Java programs run AI models directly on your own computer's hardware, instead of calling a cloud API. It brings together several kinds of AI in one package: text generation from large language models, speech-to-text transcription using Whisper style models, text-to-speech, and understanding images, audio, and video, all through a single shared engine. The project is built on top of two existing open-source engines, GGML and llama.cpp, which do the actual heavy computation. What libargus adds is a carefully designed bridge between that C++ engine and Java, using a newer Java feature called the Foreign Function and Memory API. This bridge is built to avoid unnecessary copying of data and to avoid triggering Java's automatic garbage collector while data is flowing through, which matters for performance when processing large amounts of text, audio, or video frames. It also shares memory and hardware resources across all these AI tasks at once, rather than each one loading its own separate copy of the model engine. For a video, for example, the library can decode it frame by frame using FFmpeg running as a subprocess, and hand those frames to a vision model along with timestamps. For text generation, it supports advanced techniques like speculative decoding, where the model drafts several possible next words ahead of time to speed up generation, and it lets you compress the model's working memory to save space. To use this project, you need a C++ build toolchain and CMake to compile the native library, which can also be built with CUDA support for running on an NVIDIA GPU. Once built, Java 22 or newer is required to use the provided Java bindings, since they depend on the newer Foreign Function and Memory API. This is aimed at developers building AI applications who want tight control over performance and memory rather than an easy drop-in tool. The README does not state a license.