ulyssestenn/omt

Ollama Model Tester is a simple command-line script that helps you compare AI models running locally on your machine. Ollama is a tool that lets you run AI language models (like Llama or Gemma) on your own computer rather than sending data to a cloud service. This script sits on top of Ollama and makes it easy to test how different models respond to the same question. The basic use case is: you have a task in mind, you want to know which of your installed models handles it best, so you run the same prompt through several models and save all the responses to files you can read and compare at your leisure. You can also run the same prompt through one model multiple times to see how consistent or varied its answers are, which is controlled by the temperature setting (higher temperatures produce more varied responses). Running the script asks you a few questions interactively: which model to use, what prompt to send, how many times to run it, and what temperature to use. All of those questions can also be answered via command-line flags, making it straightforward to script or automate. Results are saved in a folder named after the first few words of your prompt. All responses to the same prompt, regardless of which model generated them, land in the same folder. Each model gets its own file inside that folder containing its responses along with timing data and token counts that Ollama reports. The script requires no installation beyond Python 3.7 and a running Ollama instance. It uses only Python's built-in standard library, so there are no packages to install with pip. The README does not mention a license.