How can one neuron hide several human-readable features?

The report supplied a practical foothold for decomposing superposition: learn more sparse directions than the model has neurons, then test whether each direction activates and intervenes in a coherent way. Its strongest claim is comparative, not absolute. The learned features are more interpretable than the original neuron basis in this testbed. They can split, merge, remain partly polysemantic, and leave part of the model unexplained as dictionary width and sparsity change.

Trenton Bricken and 23 others · Transformer Circuits Thread · October 4, 2023 prototype 6 min read Explained by Superalignment Research

Read the original

Concept explainer: Mechanistic interpretability in the Superalignment wiki

Topics: mechanistic interpretability, interpretability, measurement, training data, robustness

The 60-second answer

The work turned superposition from a toy-model hypothesis into an inspectable decomposition method, while exposing resolution and coverage as first-class audit choices.

Bricken and colleagues show that a sparse autoencoder can extract interpretable directions from a small transformer's polysemantic MLP. In the main 4,096-feature run, individual directions respond to Arabic script, DNA, base64, and Hebrew, affect matching output probabilities, and often have no close neuron counterpart. A blinded human scored the median feature interval 12 versus 0 for neurons, while an automated logit test scored 74 percent versus 58 percent. Yet the learned units are resolution-dependent, not a proven ground-truth dictionary. The report studies a one-layer model and recovers 79 percent of its MLP loss contribution in the main run.

A sparse autoencoder can represent one MLP activation as a small combination of more learned directions than there are neurons.
Selected directions were specific, causally useful, and reproducible across model seeds while remaining diffuse in the neuron basis.
Human and automated tests rated learned features more interpretable than neurons, but neither test establishes a unique ground-truth ontology.
Feature identity depends on dictionary width: broad units split into narrower token-in-context units as resolution increases.

Written for: Technical generalists comfortable with vectors, neural-network activations, and basic loss functions. Useful prerequisites: A neuron produces one coordinate in an activation vector, A linear direction can combine many neuron coordinates, An autoencoder learns to reconstruct its input.

The question: Can a sparse autoencoder recover interpretable features that are hidden across polysemantic neurons in a language model?
What the authors did: The authors train two one-layer transformers whose MLP has 512 ReLU neurons, then fit sparse autoencoders to eight billion MLP activation samples from the Pile. The autoencoders reconstruct each activation with a sparse weighted sum of learned directions, using mean squared reconstruction loss, an L1 activation penalty, and resampling for dead units. Dictionary widths range from 512 to 131,072 features. The report studies selected features with computational proxies, ablation and activation interventions, then compares features with neurons through blinded human scoring, automated explanations, loss recovery, and cross-seed similarity.
The source: Towards Monosemanticity: Decomposing Language Models With Dictionary Learning

What changes when the feature dictionary gets wider?

A/1 selected. The 4,096-feature dictionary splits base64 into three related features and recovers 79 percent of the MLP loss reduction.

Reported feature resolution and reconstruction

Run	Learned features	Expansion	Base64 example	MLP loss reduction recovered
A/0	512	1x	One broad base64 feature	Not reported in the cited comparison
A/1	4,096	8x	Three related base64 features	79%
A/2	16,384	32x	Many more base64-related features	Not reported in the cited comparison
A/5	131,072	256x	Resolution continues to refine feature families	94.5%

Feature-family descriptions come from Phenomenology. Loss recovery comes from Global Analysis. The report warns that recovery is not a complete measure of interpretability.

Select a dictionary width to follow the report's base64 example and the measured loss recovery where reported. The apparent unit of explanation changes as broad features split into narrower ones.

Walk through the argument

Each step below points to the source location that carries it. The source map follows the walkthrough for source-level checking.

Start with a neuron that means too many things

One neuron in the subject model responds to academic citations, English dialogue, HTTP requests, and Korean text. Naming that neuron after any one pattern hides the others. The report argues that the neuron basis is therefore a poor unit for human understanding.

Superposition offers a mechanism. A model can represent more sparse features than it has neurons by assigning each feature a direction across many neurons. The observed activation is then a mixture of whichever directions are active on that token.

Source: Opening, Summary of Results, and Problem Setup

Learn an overcomplete sparse dictionary

The subject transformer has one attention block and one 512-neuron ReLU MLP. The sparse autoencoder receives MLP activations and learns to reconstruct each one from a small set of nonnegative feature activations and decoder directions.

Reconstruction error pulls the decoded vector toward the original activation. An L1 penalty pushes most feature activations to zero. The main run learns 4,096 features from eight billion activation samples, eight times the number of MLP neurons.

Source: Problem Setup, Using Sparse Autoencoders and Sparse Autoencoder Setup

Demand more than a plausible label

For feature A/1/3450, high activations mostly occur on Arabic script. Arabic text is 0.13 percent of training tokens but 81 percent of tokens where the feature is active. Its activation correlates 0.74 with a computational Arabic-script proxy across 40 million tokens.

The authors then ablate the feature and reduce the probability of matching Arabic tokens, or pin it high and steer generation toward Arabic text. A counterpart learned from another model seed has 0.91 activation correlation. No single neuron carries the same clean pattern.

Source: Detailed Investigations of Individual Features, especially Arabic Script Feature, Arabic Script Feature, Feature Downstream Effects, ablation and pinned sampling, Phenomenology, Universality

Move from examples to the typical feature

The detailed Arabic, DNA, base64, and Hebrew cases are cherry-picked because they have simple proxies. For a broader test, one blinded author scored 412 activation intervals across 162 features and neurons. The median feature interval scored 12 on the rubric, while the median neuron scored 0.

Automated tests point the same way. Claude predicted held-out feature activations better than neuron activations. When asked whether an explanation matched unseen positive-logit tokens, it averaged 74 percent for features and 58 percent for neurons. These tests measure interpretability under their rubrics, not truth of the labels.

Source: Global Analysis, How Interpretable is the Typical Feature

Turn the dictionary-resolution dial

At 512 learned features, the report finds one broad base64 feature. At 4,096, it splits into three: one favors letters, one digits, and one base64 strings that encode ASCII. At still larger widths, many more related features appear.

The refinement is not always a clean tree. Features can split and merge between adjacent runs. A useful explanation must therefore include the model, layer, dictionary width, and sparsity setting. Without that provenance, two researchers can use the same feature name for different units.

Source: Phenomenology, Feature Splitting and Features which seemed like Bugs

Keep coverage and scale in view

The main A/1 run recovers 79 percent of the MLP layer's contribution to log-likelihood loss. The largest A/5 run reaches 94.5 percent. More recovered loss does not prove that every learned direction is cleanly interpretable, and unexplained residuals may contain important behavior.

The testbed has one layer. A hypothetical sparse autoencoder with 100 times expansion on a width-10,000 MLP would have about 20 billion parameters. Even if decomposition scales, analyzing millions of rare features and composing them into a model-level account remains a separate problem.

Source: Global Analysis, How much of the model does our interpretation explain, Discussion and Future Work

Source map

These are the source locations that carry the argument. Use them to check this explanation against the original rather than trusting the summary alone.

Where the argument lives

Locus	Why it matters	Source
Opening, Summary of Results, and Problem Setup	Motivates polysemanticity and superposition, states the report's comparative claims, and specifies the one-layer transformer and dictionary widths.	Open source →
Problem Setup, Using Sparse Autoencoders and Sparse Autoencoder Setup	Defines the overcomplete sparse decomposition, reconstruction and L1 losses, eight-billion-sample training set, dead-unit resampling, and missing quality metric.	Open source →
Detailed Investigations of Individual Features, especially Arabic Script Feature	Tests specificity, sensitivity, downstream effects, neuron non-equivalence, and cross-seed universality for selected features.	Open source →
Arabic Script Feature, Feature Downstream Effects, ablation and pinned sampling	Shows that ablating or pinning a learned direction changes matching token predictions, establishing a causal role beyond dataset correlation.	Open source →
Global Analysis, How Interpretable is the Typical Feature	Reports dead-feature exclusions, blinded human scoring, automated activation explanations, and the 74 versus 58 percent logit test.	Open source →
Global Analysis, How much of the model does our interpretation explain	Reports 79 percent loss recovery for A/1 and 94.5 percent for A/5, then explains why loss fraction is not a complete interpretability metric.	Open source →
Phenomenology, Feature Splitting and Features which seemed like Bugs	Shows broad features splitting at larger dictionary widths, including one base64 feature becoming three and then many more.	Open source →
Phenomenology, Universality	Compares matched features across two independently initialized one-layer transformers and reports median activation correlations for features and neurons.	Open source →
Discussion and Future Work	States uncertainty about feature ontology, local versus compositional codes, scaling cost, feature-quality measurement, and analysis bottlenecks.	Open source →

The Assumption Switch

One result. One assumption exposed. Turn it and see what changes.

Assumption under test

A feature learned at one dictionary width is the stable atomic unit a researcher should explain.

Held in the source: The report focuses on A/1, an eightfold expansion with 4,096 learned features, where the base64 context is divided among three related features.
Turn it: Use a coarser 512-feature dictionary or a finer dictionary with 16,384 to 131,072 features while holding the subject model fixed.
What changes: One broad feature can split into several context-sensitive features, and features at adjacent resolutions can both split and merge. Interpretations must record dictionary scale rather than present one basis as the unique ontology.

The common misreading

The report did not discover one final list of the model's true concepts or solve mechanistic interpretability. It found a useful sparse basis for one MLP testbed and showed several causal, reproducible features. The authors explicitly say the correct number of features may be ill-posed and that scaling both autoencoders and analysis remains open.

Outside the ML frame

Cartography and measurement theory

When does changing the map scale change the objects that appear on the map?

A coarse map groups local roads into one corridor, while a fine map separates junctions and side streets. The report's feature splitting behaves similarly: dictionary width changes which regularities count as one unit without changing the underlying model. This analogy is our interpretation, but it captures the report's warning that useful decompositions can exist at several resolutions.

Where the result stops

The subject is a weak one-layer transformer with one 512-neuron MLP, so transfer to deep frontier systems was not tested. The four detailed feature cases are deliberately easy to proxy and cherry-picked. Human scoring covered 412 activation intervals across 162 features and neurons, with one blinded author as annotator. Automated interpretability uses Claude both to explain and score predictable effects. The authors lack a trusted global metric for feature quality. A/1 has 168 dead and 292 ultralow-density features excluded from analysis. Low activations are less interpretable, loss recovery is incomplete, and different dictionary widths produce different decompositions.

The numbers, with their measurands

Each value below states its measurand, evidence type, source location and evidence base when the source reports one. These fields distinguish self reported results from independent measurements.

512 neurons to 4,096 features. subject MLP width and learned feature count in the main A/1 sparse autoencoder run. Reported as self reported, Problem Setup and Notation for Features. Evidence base: one one-layer transformer and one main dictionary run. Check it →
12 versus 0. median blinded human interpretability-rubric score for feature intervals versus neuron intervals. Reported as self reported, Global Analysis, Manual Human Analysis. Evidence base: 412 activation intervals across 162 features and neurons. Check it →
74% versus 58%. automated accuracy for predicting whether unseen logit tokens matched feature versus neuron explanations. Reported as self reported, Global Analysis, Automated Interpretability, Logit Weights. Evidence base: all eligible learned features and neuron baselines in the automated logit analysis. Check it →
79% and 94.5%. MLP log-likelihood loss reduction recovered by A/1 and A/5 respectively. Reported as self reported, Global Analysis, How much of the model does our interpretation explain. Evidence base: two sparse autoencoder runs on the one-layer A transformer. Check it →
0.72 versus 0.46. median best-match activation correlation across model seeds for features versus neurons. Reported as self reported, Phenomenology, Comparing features between two one-layer transformers. Evidence base: A/1 features matched to B/1 features and neurons matched across the two models. Check it →

What remains open

Which feature-quality metric predicts successful causal use without relying on a human-readable label?
How should sparse autoencoder width and sparsity scale with model width, depth, and training data?
Can a decomposition stay stable across model checkpoints, architectures, datasets, and random seeds?
How much safety-relevant behavior remains in reconstruction error or low-activation polysemantic tails?
Can local feature explanations compose into a faithful account of a multilayer model's behavior?
How can independent auditors inspect millions of rare features without delegating the entire judgment back to another model?

How it sits against other work

extends Toy Models of Superposition

Source audit and review status

What was checked, and against what

Field	Result	Checked	By	Against
`title`	exact	2026-08-17	codex-primary-source-review	source
`authors`	exact	2026-08-17	codex-primary-source-review	source
`date`	exact	2026-08-17	codex-primary-source-review	source
`full_text`	exact	2026-08-17	codex-primary-source-review	source

Review status prototype.
Program collection seminal v1; mechanism study; wave 4, release slot unassigned.
Source access public full text, HTML. Open the reading copy →
Provider-authored safety claim yes.
Explained by Superalignment Research.
Reviewed by No named human reviewer yet.
Explainer dates created 2026-08-17; updated 2026-08-17.
AI assistance AI assisted with primary-source retrieval, full-report and appendix extraction, interactive feature-browser review, source-locus checking, first-pass prose, figure design, and implementation. The page remains a prototype until a named human review is recorded.
Rights and access The full Transformer Circuits report, appendix, and feature browsers are publicly accessible. The page states a CC BY 4.0 license for the report, subject to its own terms and asset notices.
Corrections Read the correction policy or report an error.

Provenance

First seen 2026-08-16, via hand-written example, cycle 1.
Work id work:towards-monosemanticity, which groups manifestations of the same intellectual work.
Record id url:transformer-circuits-monosemantic-2023, the natural key for this catalog manifestation.
2026-08-16 created as the verified-tier worked example
2026-08-17 full publisher report, appendix, and feature interface read; implementation-ready Explained prototype prepared

Full audit data, including this record under id url:transformer-circuits-monosemantic-2023: /library/records.jsonl. Compact browser index: /library/corpus.json. Catalog method and counts: /library/index.json.