tonyxichen/oasr

OASR is the official code release for the ICML 2026 paper All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMs. It accompanies research into how large language models implement specific tasks internally. Researchers studying LLMs often look for circuits, small subsets of connections inside the model responsible for a particular behavior. A common assumption, the Functional Anisotropy Hypothesis, is that each task maps to one unique or nearly unique circuit. This paper challenges that by showing empirically that the same task can be handled by multiple distinct circuits sharing almost no internal structure, yet each working faithfully on its own. To find these competing circuits systematically, the paper introduces Overlap-Aware Sheaf Repulsion, or OASR, a method that adds a penalty discouraging newly found circuits from reusing edges already identified in previous runs. This surfaces structurally different solutions rather than rediscovering the same one. The experiments run across several circuit discovery algorithms, ACDC, EAP, Edge Pruning, and DiscoGP, and benchmark tasks including IOI, BLiMP, and Docstring. One notable finding is an ultra sparse three edge circuit for the IOI task that reaches 86.7 percent accuracy on its own, yet none of its three edges turns out to be strictly required once the task is broken into its underlying template variants. The paper explains this by proposing that many low overlap, faithful circuits can exist for the same behavior once a model represents information in a high dimensional, overlapping way. The repository contains Python implementations of all four algorithms, shared utility code for loading models and datasets, Jupyter notebooks that reproduce each paper experiment, and saved circuit artifacts. It requires Python 3.10 or later, with PyTorch and the transformer-lens library as key dependencies. Supported models include GPT-2 small and Pythia. The full README is longer than what was shown.