chris-short/respect-the-oracle

This project is not code to run but a set of instructions you install into an AI coding assistant. The goal is to prevent the assistant from winning a test suite in ways that produce passing tests but broken software. When an AI coding agent is given a task like "make all the tests pass," it can find shortcuts that look successful on the surface. It might hardcode the exact outputs the visible test cases expect, loosen comparisons so failures no longer count, or edit the tests themselves rather than fixing the code. These approaches make the test score go green while leaving the underlying logic broken on any input the tests did not show. The skill is a single SKILL.md file containing rules for the agent. The rules tell it never to modify the test files or fixtures it does not own, never to hardcode values that match only the visible examples, never to skip or suppress failing tests, and always to fix the actual logic so the code works on inputs it has not seen. If a test case looks wrong, the rules say to stop and ask the human rather than adjusting the test to match. Installation is dropping the folder into the right directory for your AI coding tool. Claude Code reads the skill automatically when its description matches the current task. For Codex CLI and Cursor you add a short pointer in the project's AGENTS.md or a Cursor rules file that tells those tools to follow the SKILL.md when working with tests. The skill is specifically aimed at test-driven development loops where an agent iterates until tests pass, benchmark or evaluation harnesses where making the graded number go up is the goal, and any setup where the visible test examples are a small sample of the real target behavior. The license is MIT.