Superalignment

Wiki · Updated August 14, 2026

Mechanistic interpretability

The research field that reverse-engineers neural networks into human-understandable mechanisms, pursued as a verification channel that does not rely on model behavior.

By the Superalignment Team · Last reviewed September 8, 2026

In plain terms

A neural network computes with millions of interacting parts nobody designed individually. Mechanistic interpretability tries to reverse-engineer those parts into mechanisms a person can follow, the way an anatomist works out organs from tissue. The safety stake: it might let you check what a model will do by looking inside it, instead of trusting what it shows you.

Mechanistic interpretability is the research field that attempts to reverse-engineer trained neural networks into human-understandable components and circuits. Within AI safety it carries a specific hope: a verification channel that does not route through the model's behavior or self-report, and therefore does not inherit the failure modes of behavioral evaluation or the eliciting latent knowledge bound.

Origins

The modern field grew from the circuits program led by Chris Olah, announced as a research agenda in Zoom In: An Introduction to Circuits (Distill, 2020): treat individual neurons and their connections the way biology treats cells, and reverse-engineer vision models feature by feature, curve detectors, car detectors, assembled into legible circuits. Applied to transformers, the program produced its first celebrated mechanism in induction heads (Olsson et al., 2022), attention structures implementing copy-and-continue pattern matching that emerge in a discernible phase change during training and account for much of in-context learning at small scale: an existence proof that nontrivial model behavior can be traced to identified internal mechanisms.

The superposition obstacle and the SAE era

The program's central obstacle got its formal treatment in Toy Models of Superposition (Elhage et al., 2022): networks pack more learned features than they have neurons by storing them in overlapping directions, so most neurons respond to unrelated mixtures of concepts and are individually meaningless. Interpretability's unit of analysis had to move from neurons to directions, and the tool for finding them became the sparse autoencoder (SAE), trained to decompose activations into many sparsely active features.

Towards Monosemanticity (Anthropic, 2023) demonstrated the decomposition cleanly at small scale. Scaling Monosemanticity (May 2024) extracted millions of features from a production model, Claude 3 Sonnet, including features tracking deception-adjacent concepts, security vulnerabilities, and bias, and demonstrated causal control by clamping features, publicized as Golden Gate Claude, a version of the model steered into obsessive bridge-reference by one amplified feature. OpenAI's Superalignment-descended interpretability team scaled the same tool to GPT-4 with a 16-million-latent autoencoder (Gao et al., 2024) and open-sourced its Transformer Debugger. From 2025, Anthropic's attribution graphs (On the Biology of a Large Language Model, March 2025) moved from cataloging features to tracing computations through them, producing step-by-step accounts of how the model plans rhymes, performs arithmetic, and decides to refuse, with circuit updates continuing through 2026.

The safety bet, stated and contested

Dario Amodei's The Urgency of Interpretability (April 2025) states the bet at full strength: build "an MRI for AI" before models that out-think their supervisors arrive, with a stated goal that "interpretability can reliably detect most model problems" by

  1. The essay's framing is a race between interpretability's maturity and the arrival of what it calls a country of geniuses in a datacenter.

The bet is genuinely contested inside the field, and the clearest evidence is a split between its two best-resourced practitioners. Google DeepMind's alignment team publicly deprioritized SAE research in 2025-2026 after what it described as primarily negative results, SAE-based tools underperforming simpler baselines on downstream tasks, shifting toward cheap production probes for Gemini, chain-of-thought monitoring, and behavioral auditing tools. Anthropic doubled down on circuits. Both sides agree a non-behavioral verification channel would be decisive; they disagree about whether this one can be built in time, which is the empirically honest version of the argument.

A third strand automates the microscope: Anthropic's auditing agents (2025) and the open-source Petri tool (October 2025) apply models to the interpretability and auditing of models, with the same who-checks-the-checker structure as every scalable oversight scheme.

Why it matters for superalignment

Every oversight protocol that consumes model outputs inherits the possibility that outputs and internals diverge: alignment faking is that divergence demonstrated behaviorally, and eliciting latent knowledge is its worst-case formalization. Interpretability is the main research bet on reading the internals directly. If inspection works, it changes what a safety case can claim: trustworthiness arguments currently blocked on "we cannot verify what the model will do" become available, which is why Anthropic's published ASL-4 safety case sketches lean on mechanistic evidence and why the field's own skeptics keep working on probes, the cheaper cousin, even while doubting the full program.

Limitations

No current interpretability method verifies a frontier model's alignment. The specific gaps are named in the field's own publications: feature-level findings do not yet compose into whole-behavior guarantees; SAE decompositions leave reconstruction error and miss features, so absence of evidence in the dictionary is not evidence of absence in the model; probe reliability degrades off-distribution, exactly where verification is needed most; and attribution-graph analyses cover selected behaviors of selected models at meaningful analyst cost. Chain-of-thought monitoring, the pragmatic fallback, depends on reasoning staying legible, which a 40-author position paper argued is real but fragile under outcome-based training. The honest status is a promising instrument with an undetermined arrival date, racing a capability curve that does not wait, with the field's largest practitioners disagreeing in public about whether the current main tool is the right one.

FAQ

Can interpretability verify that a model is aligned today?

No. Current methods can identify features and circuits for specific concepts, causally steer some behaviors, and flag some deception-related internals, in studied models, for studied behaviors. No method today provides whole-model guarantees, and the field's stated goals, such as reliably detecting most model problems by 2027, are targets, not achievements.

What is the disagreement about sparse autoencoders?

Sparse autoencoders decompose a model's internal activations into interpretable features. Anthropic scaled them to production models and built its circuit-tracing program on top. Google DeepMind's alignment team reported primarily negative results, finding SAE-based tools beaten by simpler methods on practical tasks, and shifted resources toward lightweight probes and chain-of-thought monitoring. The split is about tool choice and timelines, not about whether internal verification would matter.

Sources