nopainnullgain/localeyes

LocalEyes gives sight to text only AI coding models, such as DeepSeek or CodeLlama, that are otherwise unable to see screenshots, error dialogs, or user interfaces. It does this by pairing the text model with a separate vision model that runs locally on your own computer through a tool called Ollama, so no images are ever uploaded to a cloud service. Normally, if a text only model needs to understand something visual, a common workaround is uploading a screenshot to a different cloud based model like GPT-4 or Gemini and asking it to describe what is on screen. LocalEyes avoids this by running its own local vision model, so the screenshot never leaves your machine and the coding assistant's workflow does not need to be interrupted. Setup involves downloading a specific vision model called qwen2.5vl through Ollama, installing a Python image library, cloning the repository, and running a small install script. Once set up, there are two ways to use it. In one, you take a screenshot yourself and ask the model a question about it. In the other, the text based model decides on its own, during its normal work, that it needs to look at the screen, and it runs the vision tool itself, for example after finishing a build or while debugging. Under the hood, the tool captures an image from the screen, clipboard, or a saved file, sends it to the local Ollama vision model running on the same machine, and gets back a written description that the text based model can then read and reason about. The vision model and its settings, such as which model to use and how long to wait for a response, can be changed in a configuration file or through environment variables. LocalEyes has been fully tested on Windows and is expected to work, though untested, on macOS and Linux, with some extra setup needed for the Wayland display system on Linux. It is written in Python and released under the MIT license.