joshhu/skillopt-qa

This repository is a small, faithful reimplementation of Microsoft's SkillOpt, a research technique for improving AI agents without touching the underlying model. Instead of retraining a language model's internal weights, which is expensive and slow, SkillOpt trains a reusable block of plain natural language text called a skill file, which gets pasted in front of the agent's instructions to make it perform better on a task. This version is built specifically for HotpotQA, a benchmark of multi-step reasoning questions where the answer requires connecting facts from more than one source. The training process borrows ideas from neural network training, using rounds called epochs and small batches, but the thing being updated each round is text, not numbers. After each batch, a second AI model proposes a small, targeted edit to the current skill text. That candidate skill is tested on a held out validation set, and the edit is only kept if it actually improves accuracy, otherwise it is rejected and the optimizer is told not to repeat that idea. This validation gate is meant to prevent the skill from just memorizing the training examples. The only thing produced at the end of training is a single file, best_skill.md, which can be deployed with any compatible chat model by attaching it to that model's system prompt. To run it, you need Python 3.10 or newer, the uv package manager, and access to an OpenAI-compatible chat API endpoint, no GPU-specific packages are required, since the actual model runs elsewhere and this project just talks to it over the network. The default configuration expects a local Qwen model served through a compatible API. The project includes command line tools for downloading the HotpotQA dataset and for running the full training loop, along with a test suite that runs entirely offline using a fake model. The README includes a results table showing that on one experimental run, the optimized skill improved both validation and held out test scores over the seed skill, without changing any model weights, though the authors note the improvement was modest since the base model was already strong on this task.