mudler/localai

LocalAI is a self-hosted, open-source server that lets you run AI models on your own hardware and access them through an API that is compatible with the OpenAI API format. The goal is that any application built to work with OpenAI's paid cloud API can be pointed at a LocalAI instance instead, with no code changes, while all processing happens locally, meaning your data never leaves your infrastructure. The server supports a wide variety of AI capabilities beyond text generation: vision (analyzing images), voice (speech recognition and text-to-speech), image generation, and video generation. It connects to over 36 different AI backends under the hood, engines like llama.cpp, Whisper, diffusion models, and vLLM, automatically selecting the right one based on the model you load and the hardware you have. A key selling point is hardware flexibility. LocalAI works on NVIDIA, AMD, and Intel GPUs, Apple Silicon, and even runs on CPU alone when no GPU is available. Models can be loaded from a built-in gallery, from Hugging Face, from Ollama's model registry, or from configuration files. The tool detects your hardware and downloads the appropriate backend variant automatically. Beyond the core API server, LocalAI includes multi-user access control with API keys and quotas, built-in AI agents that can call external tools, and support for RAG (retrieval-augmented generation, a technique that lets a model answer questions using content from documents you provide). You would use LocalAI when you want the capabilities of cloud AI APIs but need data privacy, cost control, offline operation, or the ability to run open-weight models without a subscription. It is written in Go, MIT licensed, and deployable via Docker with a one-line command.