scisharp/llamasharp

LLamaSharp is a C# library that lets .NET developers run AI language models directly on their own computer or server, without sending data to a cloud service. It is built on top of a lower-level tool called llama.cpp, which handles the actual computation, and LLamaSharp wraps it with a friendlier API for .NET applications. The name refers to LLaMA, a family of open-weight AI language models originally released by Meta, though the library works with other compatible models as well. Installation comes in two parts. First you install the main LLamaSharp package from NuGet, which is the standard .NET package manager. Then you install a backend package that matches your hardware: a CPU-only backend that works on Windows, Linux, and Mac, a CUDA 11 or CUDA 12 backend for Nvidia GPUs on Windows and Linux, a Vulkan backend for other GPUs, or the CPU backend on Mac, which also uses the Metal GPU acceleration built into Apple hardware. No C++ compilation is required. Models must be in a format called GGUF. If a model you want to use is in a different format, you can convert it, but many pre-converted GGUF files are available to download directly from Hugging Face. Once set up, the library lets you load a model and generate text responses, hold multi-turn conversations, and process both text and images with multimodal models. It integrates with several Microsoft tools: Semantic Kernel, which is a framework for building AI-assisted applications, and Kernel Memory, which adds the ability to index and search documents so the model can answer questions about your own content. That pattern, where a model retrieves relevant documents before answering, is called RAG. The project also works with other frameworks including LangChain for .NET and BotSharp. Example projects in the repository show integrations with ASP.NET web applications, WPF desktop apps, Blazor, and Unity. A community Discord server and a QQ group are available for questions and support.