patrickiel/pantomime

Pantomime is a testing tool for Windows applications that lets you write test instructions in plain English. Instead of scripting exact clicks or code, you describe what a user would do, something like "type the username, click Sign in, confirm the welcome screen." Pantomime then carries out those steps on the actual screen, as if a person were sitting at the computer. The tool works by combining a few pieces. A perception layer reads the screen using Windows built in accessibility tools, plus local image and text recognition for parts of the screen those tools cannot reach. A reasoning model, powered by an AI API you provide, plans each step and checks whether the result matches what you described. A driver then performs the clicks and keystrokes on whatever region of the screen is relevant, without needing to know what application it is interacting with. To use Pantomime, you need a Windows machine, Python 3.12, and an API key for a reasoning model from a provider like DeepSeek, Anthropic, or OpenAI. You install the tool from its GitHub repository, then run a command to set up a test folder in your project. Tests are written as simple YAML files. A basic test just needs an ID, a title, and a list of steps written in natural language. You can also use variables and secrets, so your tests can reference things like usernames and passwords without hardcoding them. Pantomime is not yet published to standard Python package repositories, so you install it directly from the cloned repository. Configuration lives in a single YAML file where you specify your project name, which AI models to use, how much reasoning effort the model should apply, and pricing for each model. The tool also generates schema files that give you autocomplete and validation in editors like VS Code, which helps catch typos before you run anything. The project is open source under the MIT license and welcomes contributions. The README is fairly detailed about setup and configuration, though it was cut off before fully showing test examples and the deeper internals of how the tool works. The full README is longer than what was shown.