Why can a network hold more features than dimensions?
The toy models demonstrate a concrete capacity tradeoff. When features are sufficiently sparse, the benefit of storing another feature can exceed the interference it causes, so a nonlinear network packs features into non-orthogonal directions. This offers a mechanism for polysemantic representations and a test bed for interpretability methods. It is not direct evidence that the same geometry or learning dynamics governs large language models.
Nelson Elhage and 15 others · Transformer Circuits Thread · September 14, 2022 prototype 5 min read Explained by Superalignment Research
The 60-second answer
If models use more features than dimensions, inspecting one neuron at a time can mix several concepts and miss distributed directions. Interpretability then becomes an overcomplete measurement problem: find more meaningful features than the activation space has axes, and show that the basis predicts interventions.
Elhage and colleagues show that sparse features can be represented in superposition: more features than dimensions occupy non-orthogonal directions. Interference is tolerable because sparse features rarely co-occur, a negative bias can suppress small cross-talk, and ReLU removes negative interference. As sparsity changes, the toy networks move sharply among not representing a feature, assigning it a dedicated dimension, and packing it with others. The learned directions often form regular geometric structures and can support simple computation in superposition.
- Sparse features can share non-orthogonal directions because they rarely activate together, trading extra capacity for occasional interference.
- The representation changes sharply among dropped, superposed, and dedicated features as sparsity and importance change.
- The experiments establish a mechanism in tractable toy networks, while evidence for the same geometry in large models remains incomplete.
Written for: Technical generalists comfortable with vectors, neural networks, and ReLU activations. Useful prerequisites: Vectors, dot products, and linear projections, Autoencoders, ReLU, and mean squared error.
- The question
- When can a neural network represent more sparse features than it has available dimensions?
- What the authors did
- The article trains small autoencoder-like networks on synthetic features with controlled sparsity and importance. Inputs with more features than hidden dimensions are encoded linearly and decoded with tied weights, with either a linear output or a ReLU output and bias. The authors vary sparsity, feature importance, correlation, and computation, then analyze learned loss, geometry, phase changes, and packed circuits against tractable theoretical cases.
- The source
- Toy Models of Superposition
When is packing three features into two dimensions worth it?
Dense mode selected. Co-activation is common, so the interference cost can outweigh the capacity gained from packing another feature.
| Feature regime | Interference | Representation pressure |
|---|---|---|
| Dense | Packed features co-activate often. | Favor dedicated dimensions for the most important features. |
| Sparse | Packed features collide only on uncommon inputs. | Extra feature capacity can justify non-orthogonal directions. |
ReLU and negative bias can filter part of the cross-talk, but superposition still pays a real interference cost.
Walk through the argument
Each step below points to the source location that carries it. The source map follows the walkthrough for source-level checking.
Separate a feature from a neuron
A neuron is one coordinate in an activation space. A feature is a property of the input that the network finds useful. A feature can align with one coordinate, spread across several coordinates, or share a direction with other features.
The article asks whether networks represent features as directions even when there are more useful features than dimensions. This makes a polysemantic neuron a possible symptom of crowded geometry rather than proof that the neuron is the fundamental unit.
Build a bottleneck with known ground truth
The toy input has named features whose importance and sparsity are controlled. An encoder projects them into fewer hidden dimensions. A tied decoder reconstructs the features, and a ReLU plus bias can remove negative or small outputs.
With a purely linear decoder, the best solution resembles principal component analysis and keeps the most important features in orthogonal directions. Nonlinearity changes the bargain because some cross-talk can be filtered after decoding.
Source: Publisher PDF pages 8 to 13, Basic Results and Mathematical Understanding
Pay interference only when features collide
Think of three intermittent radio channels sharing two frequencies. If only one usually transmits, sharing can work. If all transmit together, each signal corrupts the others. Sparse neural features make the same capacity tradeoff.
A new non-orthogonal feature improves reconstruction when it appears alone and creates interference when packed features co-occur. ReLU removes negative interference, while a negative bias can suppress small positive noise. The optimum balances benefit against collision cost.
Source: Publisher PDF pages 8 to 13, Basic Results and Mathematical Understanding, Publisher PDF pages 16 to 25, Geometry of Superposition
Watch representation change by phase
In a two-feature, one-dimension case, the authors can compare several exact strategies. A feature may be dropped, share the dimension with another feature, or take the dimension alone. As sparsity changes, the lowest-loss strategy switches sharply.
This explains why training curves and feature geometry can jump instead of changing smoothly. A small shift in sparsity or importance can move the model across a boundary where an entirely different representation wins.
Source: Publisher PDF pages 14 to 15, Superposition as a Phase Change, Publisher PDF pages 26 to 30, Learning Dynamics, Correlated Features, and Adversarial Examples
Packed features form structured geometry
With several equally important features, learned directions often resemble regular polytopes. The geometry is not decoration. Angles determine interference, so spreading directions evenly can reduce the worst collisions for a given capacity.
The article also packs simple computations, not only static features. Small absolute-value circuits can share dimensions, and asymmetric arrangements can use inhibition to control interference. This suggests that superposition may crowd algorithms as well as representations.
Source: Publisher PDF pages 16 to 25, Geometry of Superposition, Publisher PDF pages 31 to 39, Privileged Bases and Computation in Superposition
Demand a bridge to real models
The broad capacity argument is plausible whenever useful features are sparse, but exact polytope geometry and toy learning dynamics are more fragile. Real networks have correlated concepts, attention, normalization, residual streams, and no supplied feature dictionary.
The paper therefore motivates three strategies: prevent superposition, recover an overcomplete feature basis, or combine both. Any claimed basis should predict model behavior and interventions. A visually appealing decomposition alone does not establish that its features are causal or complete.
Source: Publisher PDF pages 26 to 30, Learning Dynamics, Correlated Features, and Adversarial Examples, Publisher PDF pages 40 to 50, Strategic Picture, Discussion, Related Work, and Open Questions, Publisher PDF pages 51 to 62, Comments, replications, notes, references, and nonlinear compression appendix
Source map
These are the source locations that carry the argument. Use them to check this explanation against the original rather than trusting the summary alone.
| Locus | Why it matters | Source |
|---|---|---|
| Publisher PDF pages 1 to 7, Overview and Background | Defines features, directions, polysemanticity, and the hypothesis that networks represent more features than available dimensions. | Open source → |
| Publisher PDF pages 8 to 13, Basic Results and Mathematical Understanding | Specifies the sparse-feature toy model, tied encoder and decoder, ReLU loss, and the basic capacity versus interference result. | Open source → |
| Publisher PDF pages 14 to 15, Superposition as a Phase Change | Analyzes the sharp transition among dropped, superposed, and dedicated feature representations in a tractable two-feature case. | Open source → |
| Publisher PDF pages 16 to 25, Geometry of Superposition | Shows learned regular-polytopal arrangements, dimensions per feature, and how feature importance changes the packed geometry. | Open source → |
| Publisher PDF pages 26 to 30, Learning Dynamics, Correlated Features, and Adversarial Examples | Examines abrupt changes during learning, feature correlations, and the tentative relationship between superposition and adversarial examples. | Open source → |
| Publisher PDF pages 31 to 39, Privileged Bases and Computation in Superposition | Extends the representational account to small packed circuits, including absolute-value computation and asymmetric interference. | Open source → |
| Publisher PDF pages 40 to 50, Strategic Picture, Discussion, Related Work, and Open Questions | Compares three interpretability strategies, separates strong from tentative claims, and states the main generalization limits. | Open source → |
| Publisher PDF pages 51 to 62, Comments, replications, notes, references, and nonlinear compression appendix | Records independent replications, author contributions, technical notes, and an additional nonlinear compression example. | Open source → |
The Assumption Switch
One result. One assumption exposed. Turn it and see what changes.
Assumption under test
Features are sparse enough that most packed features are not active at the same time.
- Held in the source
- When co-occurrence is rare, non-orthogonal feature directions create interference only on a small share of inputs. ReLU and a negative bias can filter some of that cross-talk, so added capacity can be worth the loss.
- Turn it
- When features are dense, packed directions interfere on most inputs. The loss from cross-talk can exceed the value of representing extra features, favoring orthogonal directions for the most important features.
- What changes
- The same bottleneck changes from a multiplexed code to a more conventional basis. Sparsity is the capacity switch, not an incidental dataset detail.
The common misreading
Superposition does not mean a model can store unlimited features for free. Each packed feature creates interference, and dense or highly important features often receive dedicated dimensions. Nor does one polysemantic neuron prove the full superposition hypothesis, because features may be represented across directions rather than aligned with individual neurons.
Outside the ML frame
Statistical multiplexing in communication networks
How can many intermittent users share less capacity than their simultaneous peak demand?
A network can serve many bursty users on shared infrastructure because they rarely demand peak bandwidth together. Congestion appears when their bursts overlap. Superposition makes a related bargain inside a representation: sparse features share dimensions, and simultaneous activation produces interference. The analogy clarifies the capacity tradeoff, though the paper studies learned geometry rather than packet scheduling.
Where the result stops
The models use synthetic independent features with known ground truth, tiny bottlenecks, tied weights, and simple losses. Several geometric results depend on the chosen feature distribution and ReLU architecture. The authors describe the geometry and learning dynamics as less likely to generalize than the broad capacity argument. Evidence from real networks is suggestive rather than decisive, and an overcomplete interpretation of a large model remains hard to identify or validate.
What remains open
- Which observations in large language models can distinguish genuine feature superposition from another cause of polysemantic activations?
- Can an overcomplete feature basis be recovered with falsifiable tests rather than chosen by an interpretability method's inductive bias?
- How do attention, normalization, and residual streams change the sparsity threshold for useful superposition?
- Can training interventions reduce dangerous interference without erasing useful rare features or increasing total loss?
What this bears on
Superalignment maintains a public register of the claims it makes and the evidence that would change its mind. The entries below connect this source to the exact public claims it bears on.
- C4. Behavioral evaluation cannot carry a deployment decision alone. This record bears on it, indirectly. The toy models show that behaviorally useful features can occupy non-orthogonal, polysemantic representations that are not identified by output tests or single-neuron inspection alone. See the claim and what would change our mind →
Source audit and review status
| Field | Result | Checked | By | Against |
|---|---|---|---|---|
title | exact | 2026-08-17 | codex-primary-source-review | source |
authors | exact | 2026-08-17 | codex-primary-source-review | source |
date | exact | 2026-08-17 | codex-primary-source-review | source |
full_text | exact | 2026-08-17 | codex-primary-source-review | source |
- Review status prototype.
- Program collection seminal v1; mechanism study; wave 4, release slot unassigned.
- Source access publisher full text, Interactive HTML and PDF. Open the reading copy →
- Provider-authored safety claim yes.
- Explained by Superalignment Research.
- Reviewed by No named human reviewer yet.
- Explainer dates created 2026-08-17; updated 2026-08-17.
- AI assistance AI assisted with canonical-source retrieval, full-text extraction, page rendering, section-by-section reading, locus checking, first-pass prose, figure design, and implementation. The page remains a prototype until a named human review is recorded.
- Rights and access The complete article is publicly readable in an interactive publisher HTML edition and as a 62-page publisher PDF. A separate reuse license was not identified on the article page during this review. The AI Alignment Forum record is treated only as a same-work forum crosspost, not as an independent source.
- Corrections Read the correction policy or report an error.
Provenance
- First seen 2026-08-16, via seed_library.py, blogs shard of the Stampy snapshot.
- Work id
work:toy-models-of-superposition, which groups manifestations of the same intellectual work. - Record id
url:transformer-circuits.pub/e803b05526, the natural key for this catalog manifestation. - 2026-08-16 seeded from the blogs shard
- 2026-08-17 full publisher HTML and PDF read, same-work crosspost reconciled, and implementation-ready Explained prototype prepared
Full audit data, including this record under id
url:transformer-circuits.pub/e803b05526:
/library/records.jsonl.
Compact browser index:
/library/corpus.json.
Catalog method and counts:
/library/index.json.