recursive-org/first-steps-toward-automated-ai-research

Recursive is a company building systems that automate AI research. This repository collects code artifacts produced by their automated system, released alongside a blog post describing the project. The repository has three sections. The first covers a speedrun challenge for training a small language model called NanoGPT to a target accuracy level as quickly as possible. The automated system found an approach that reaches that target in 77.3 seconds on 8 high-end NVIDIA H100 GPUs, which the README says is faster than an earlier record holder on the same hardware configuration. The official leaderboard timing was still pending at the time of publication. The second section contains 10 example GPU kernel implementations out of 235 that the automated system wrote for NVIDIA's SOL-ExecBench competition, a leaderboard for optimizing GPU compute kernels. The other 225 implementations are kept private to avoid influencing the leaderboard. The third section contains training scripts for NanoChat, a small language model training setup, along with results from 10 separate training runs on a single GPU within a 5-minute time budget per run. None of these outputs were written by hand. The purpose of the repository is to show what an automated research system can discover: functioning, competitive code produced through machine-driven experimentation rather than human authorship. The repository releases the outputs, not a description of how the underlying automated research system itself works. The code is licensed under Apache 2.0. Two subdirectories include modified versions of MIT-licensed open-source projects, and their original copyright notices are preserved alongside the derived code.