ovachiever/agent-state

Agent state is a tool for developers who use AI coding assistants like Claude Code or Codex every day and suspect the model quietly gets worse over time. Many people notice a coding model feels sharp when it launches, then a few weeks later starts missing instructions, leaving unfinished work, or agreeing with wrong feedback instead of pushing back. This project tries to measure that feeling instead of just trusting it. Each night, the tool runs 18 coding tasks through the actual command line tools you use, in a fresh throwaway workspace, and scores the results against roughly 300 automated checks. It tracks five things: whether the code is correct, whether the model finishes multi step edits, whether it leaves stubs or half done work, whether it can hold dozens of scattered requirements across a long brief, and whether it can tell the difference between a true bug report and a confidently wrong one. That last category also checks whether the model caves when told its correct work is wrong, and whether it catches its own overclaiming when it says a task succeeded but the checks say otherwise. Alongside the daily coding battery, a fixed set of thirty math competition problems runs against the raw provider APIs every night as a steady reference point, so a drop there points to a change in the model itself rather than a tooling update. For the first two weeks after setup, the tool's verdicts stay hidden while you log your own daily gut feeling about the model's performance. At the end, the two records are compared: if the tool's hidden verdicts matched your gut, it proves itself trustworthy going forward. If not, it automatically turns off its own schedule. Setup happens through Python's uv tool, with a config file for model names, binary paths, and dates. The project is written in Python and has one star, so it is early and largely a personal project by its sole listed contributor.