norika1207-lab/llm-neuron-atlas

★ 14HTMLAudience · researcherComplexity · 2/5Setup · easy

Mindmap

mindmap
  root((Neuron Atlas))
    What it does
      3D neuron visualization
      Layer by layer exploration
      Signal flow tracing
    Supported models
      Qwen 2.5 3B main model
      Phi-3 side by side
      Mistral 7B side by side
    Named neurons
      INHIBITOR dim 715 negation
      CONTROLLER dim 758 style
      Cross layer signal paths
    Tech stack
      Single HTML file viewer
      three.js 3D graphics
      Python bake script
    Extend it
      Llama model support
      GPT-2 model support
      Custom model families

mindmap root((Neuron Atlas)) What it does 3D neuron visualization Layer by layer exploration Signal flow tracing Supported models Qwen 2.5 3B main model Phi-3 side by side Mistral 7B side by side Named neurons INHIBITOR dim 715 negation CONTROLLER dim 758 style Cross layer signal paths Tech stack Single HTML file viewer three.js 3D graphics Python bake script Extend it Llama model support GPT-2 model support Custom model families

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Explore which neurons in Qwen 2.5 3B activate for negation or style control by clicking through the 3D layer viewer in your browser

USE CASE 2

Run the Python bake script on a Llama, Mistral, or Phi-3 model to generate a neuron atlas for your own interpretability research

USE CASE 3

Compare neuron behavior across Qwen, Phi-3, and Mistral 7B side by side to look for neurons that serve similar roles across architectures

Tech stack

HTMLJavaScriptthree.jsPython

Getting it running

Difficulty · easy Time to first run · 5min

The viewer is a single HTML file that loads in under 10 seconds, running the bake script to generate data for a new model requires Python and takes longer.

In plain English

LLM Neuron Atlas is an interactive 3D visualization that lets you explore the internal structure of a large language model one neuron at a time. Think of it as a kind of Google Maps for an AI model: you can zoom out to see the entire 36-layer tower of the model at once, or zoom in to a single layer and click on individual neurons to see how their signals flow to other neurons across the network. The tool is built around the Qwen 2.5 3B model, which is a publicly available language model with about 3 billion parameters. A Python script reads the model weights, computes which neurons most strongly influence which other neurons, and writes that relationship data into a set of JSON files. A browser-based viewer then loads those files and renders the network as a 3D structure using three.js, a popular JavaScript graphics library. The whole bake process runs on a regular laptop without a graphics card. Some neurons in the visualization are labeled with names from prior research by the same author, such as the INHIBITOR neuron at dimension 715, which appears to fire specifically when the model processes negation, and the CONTROLLER neuron at dimension 758, which influences style across many layers. Colored highways trace how a single neuron dimension carries its signal all the way from the input layer to the output. You can also compare the Qwen model side by side with Phi-3 and Mistral 7B to see which neurons seem to serve similar roles across different model architectures. The viewer is a single HTML file with no build process required. A live demo is available online and loads in under 10 seconds. The output data for the Qwen 3B model is about 35 megabytes when compressed. The bake script is designed so researchers can adapt it to other model families with small changes, and the README documents exactly which tensor names to adjust for Llama, Mistral, Phi-3, and GPT-2 style models. Mixture-of-experts models like Mixtral are not yet supported.

Copy-paste prompts

Prompt 1

I am exploring LLM Neuron Atlas in the browser, how do I read the colored highways showing a single neuron's signal path from the input layer to the output?

Prompt 2

I want to generate a neuron atlas for my own Llama model using the bake script, which tensor names do I need to change in the configuration and how long does it take on a laptop without a GPU?

Prompt 3

Help me adapt the LLM Neuron Atlas bake script to produce a connectivity map for a GPT-2 model I am studying locally

Prompt 4

I found the INHIBITOR neuron at dimension 715 in LLM Neuron Atlas, show me how to trace all the downstream neurons it influences across the 36 layers

Open on GitHub → Explain another repo

← norika1207-lab on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.