yogiadhik/skillcheck

skillcheck is a testing tool for AI agent skills, the kind used with coding assistants like Claude Code. The core problem it addresses is that skills tend to fail silently. A small edit to a description or instruction can quietly break a skill, and you only discover it weeks later in someone else's session. skillcheck brings the idea of a test suite, which software code has had for decades, to these agent skills. The tool offers two main commands. lint is free and instant. It checks for problems like broken frontmatter, weak descriptions that fail to trigger the agent, broken links, and dangerous patterns. run is the behavioral test. It copies a fixture into a temporary workspace, mounts your skill, runs the agent without user prompts, and reports what actually happened, including which files changed and which commands ran. Writing a test involves creating a YAML file next to your skill. You can set deterministic checks, like confirming a specific command ran or a file changed. For fuzzier outcomes, you can include a judge rubric. The tool sends the transcript and your rubric to a cheap AI model, which scores the result on a scale, passing at seven out of ten by default. An init command can scaffold a starter test file for you. Behavioral runs cost real money because they call the real agent. To manage this, skillcheck caches results by content hash, so unchanged skills and cases do not re-run. You can also set a spending cap with a budget flag. The author has run the linter across roughly eight thousand public skill files and found common issues, including skills that never load due to invalid YAML, instructions pointing at non-existent files, and skills so large they re-bill tens of thousands of tokens on every use. The run command executes the agent inside a throwaway directory with permission prompts disabled. The author notes you should only use copies of data as fixtures, never live data, and skills you do not trust should be linted, not run. Currently the tool supports Claude Code, but it is built to allow adapters for other agents.