alikonuk1/local-relay

Local Relay is a toolkit for JavaScript and TypeScript apps that need to talk to AI models running on the user's own computer instead of a cloud service. It connects to local model runtimes like Ollama, LM Studio, llama.cpp server, vLLM, LocalAI, and other tools that speak an OpenAI-compatible API, and gives your app one consistent interface for working with any of them. With this single interface you can detect which local providers are available on someone's machine, list the models they have installed, send chat messages, stream the model's response token by token as it is generated, create text embeddings, and use OpenAI-style Responses API endpoints if the local server supports them. The library also handles a common browser problem: many local model servers block direct requests from a web page due to CORS restrictions, so Local Relay includes an optional small proxy program that runs on the user's own machine and forwards requests safely, only to local addresses. The project is split into three separate packages: a core library with the actual provider clients and logic, a set of React hooks and simple UI pieces for building model and provider pickers plus streaming chat interfaces, and the standalone proxy program itself. You only need to install the pieces you actually use. There is also a full example application built with Vite and React that shows provider detection, model selection, both direct and proxy modes, and live streaming chat working together. It is worth being clear about what this project is not: it is a connector library for wiring your own app to local models, not a finished chat application on its own. Feature support such as tool calling, image input, or embeddings still depends entirely on whether the specific local model and runtime you pick actually support that feature, since Local Relay simply forwards requests in a consistent shape rather than adding capabilities the underlying model does not have. The project is released under the MIT license, so it can be used, modified, and redistributed freely, including in commercial projects.