primeintellect-ai/experiments-autonomous-speedrunning

This repository is a raw data archive from an experiment where two AI coding agents, Claude Code and OpenAI Codex, were each given a machine learning optimization challenge and left to work on it independently over multiple rounds. The task was based on a public benchmark called modded-nanogpt: change the training recipe for a small language model so it reaches a target accuracy level using as few training steps as possible. The agents could only adjust the optimizer, the learning rate schedule, how weights are initialized, and a small set of other settings. Each agent ran many training attempts on its own, wrote out its plans and reasoning, tried new ideas based on what it had learned, and kept track of results in log files. Across three rounds of this process, called waves, both agents steadily improved. The starting reference took 3,500 training steps to hit the target. By the third wave, Claude had brought that down to 2,930 steps and Codex to 2,950 steps. Everything either agent produced is preserved here for study: planning documents, reasoning threads, generated training scripts, launch scripts, and close to ten thousand training run logs, plus notes on research papers the agents consulted and lists of candidate ideas they considered. A separate folder collects a flattened, cross-wave export of all runs with structured metadata for each one, useful if someone wants to filter or analyze the full set of results without digging through each wave's raw folders individually. This is not a tool you install or run. It is a research artifact meant for people curious about how autonomous AI agents approach open-ended optimization problems, or anyone studying AI research agent behavior. The export covers 10,428 completed training runs split across both agents and all three waves of the experiment.