Why can identical test scores hide different models?
A strong average test score does not identify which model you trained. Credible deployment therefore needs stress tests tied to the intended use, plus evidence that acceptable behavior is stable across the set of predictors the pipeline can return. Underspecification is a measurement and pipeline-design failure before it becomes an out-of-distribution failure.
Alexander D’Amour and 39 others · Journal of Machine Learning Research 23(226):1-61 · August 2022 prototype 5 min read Explained by Superalignment Research
The 60-second answer
The paper changes the unit of evaluation from one trained model to the pipeline that can produce many models. That move is central to trustworthy evaluation, reproducibility, and any safety claim intended to survive retraining or deployment shift.
D’Amour and colleagues argue that ordinary validation often identifies an equivalence class of acceptable predictors rather than one deployment-ready solution. Small pipeline changes, such as a random seed, can select different members of that class. Those models can look interchangeable on data drawn like training data and still behave differently under corruption, population transfer, shortcut tests, or clinical subgroups. The seed is a probe, not the underlying cause. The cause is that the pipeline never constrained the deployment behavior that matters.
- A pipeline is underspecified when it can return many standard-test-equivalent models that differ on deployment requirements.
- Changing a random seed probes the acceptable model set, but the missing constraint in the pipeline is the underlying problem.
- Application-specific stress tests can reveal hidden variation, but passing known tests does not prove stability under every future shift.
Written for: Technical generalists who understand train and test splits but have not studied statistical identification. Useful prerequisites: Training, validation, and test data, Basic distribution shift.
- The question
- Why can a training pipeline return models with the same standard test score but sharply different behavior under deployment stress?
- What the authors did
- The paper defines underspecification as multiplicity among predictors that satisfy the pipeline's standard validation criterion, distinguishes it from unavoidable structural conflict, and proposes retraining under small perturbations followed by application-specific stress tests. It demonstrates the pattern in simple statistical examples and in computer vision, medical imaging, natural language processing, electronic health records, and genomics.
- The source
- Underspecification Presents Challenges for Credibility in Modern Machine Learning
What changes when validation leaves deployment behavior open?
IID validation selected. All three models pass and the pipeline treats them as equivalent.
| Evaluation rule | What the pipeline sees | What the result supports |
|---|---|---|
| IID validation only | Models tie on the standard criterion. | The models satisfy that criterion, not that they are interchangeable in deployment. |
| Add deployment stress test | Previously tied models can separate. | The original pipeline left deployment-relevant behavior unconstrained. |
Random retraining is a probe of the acceptable model set. It is not the definition or root cause of underspecification.
Walk through the argument
Each step below points to the source location that carries it. The source map follows the walkthrough for source-level checking.
One test score can describe many models
Imagine a hiring exam that checks arithmetic but never tests whether candidates can explain a decision. Many people can tie on the exam and differ completely on the untested skill. Selecting one tied candidate does not make the group interchangeable.
Modern training pipelines have the same shape. They accept every predictor that meets a standard validation criterion. The paper calls the pipeline underspecified when that accepted set contains models that differ on behavior required in deployment.
Source: Version-of-record PDF pages 2 to 4, Section 1, Version-of-record PDF pages 4 to 7, Sections 2.1 to 2.3
Separate ambiguity from unavoidable conflict
A structural failure means the training criterion and deployment requirement cannot both be satisfied. Better model selection cannot fix that conflict. Underspecification is different: the accepted set contains both better and worse deployment choices, but the pipeline does not know which is which.
This distinction matters because the repair differs. Structural conflict needs a changed objective, data source, or problem formulation. Underspecification may be reduced by adding constraints or tests that choose among otherwise equivalent predictors.
Source: Version-of-record PDF pages 4 to 7, Sections 2.1 to 2.3
See the ambiguity in simple models
Early epidemic counts can identify a net growth rate without separately identifying infection rate and disease duration. Several parameter pairs fit the early curve, then imply very different later trajectories. The data answered a narrower question than the forecast requires.
In the genomics case, one thousand polygenic risk scores use different representatives from correlated feature clusters. Their British test performance is similar, while performance on a non-British population varies much more. Standard test rank weakly predicts transfer rank.
Source: Version-of-record PDF pages 7 to 11, Section 3 and Figures 1 to 3
Use retraining as a diagnostic probe
Retrain the same nominal pipeline after a small perturbation, such as a new random seed. First verify that the resulting models remain tied on standard validation. Then compare them on tests designed around real deployment requirements.
If those stress-test results spread out, the standard criterion did not determine that behavior. The seed did not create the weakness. It sampled another member of the acceptable set. Because a finite ensemble explores only a fraction of that set, the observed spread is a lower bound.
The pattern repeats across domains
Fifty ResNet-50 runs have an ImageNet accuracy standard deviation of 0.001, but the standard deviation reaches 0.024 under pixelation. Medical models differ in subgroup calibration. Language models vary in gender shortcuts and stress-test accuracy across pretraining seeds.
Electronic health record models also change rank across hospitals, years, demographic groups, and feature-removal tests. The examples do not establish one universal cause. They show that narrow validation repeatedly hides deployment-relevant degrees of freedom.
Source: Version-of-record PDF pages 13 to 16, Section 4.2, Figures 4 and 5, and Table 1, Version-of-record PDF pages 15 to 26, Sections 4.3 and 4.4, Figures 6 to 10, and Tables 2 to 4, Version-of-record PDF pages 27 to 30, Section 4.5, Figures 11 and 12, and Tables 5 and 6
What stress tests can and cannot buy
A stress test turns an unstated deployment hope into a measured requirement. Adding it to model selection can shrink the acceptable set. That is stronger than choosing the run with the highest ordinary validation score, which the paper often finds weakly related to stress-test performance.
Stress tests still cover only worlds the evaluator imagined. Model averaging is not a universal escape because ensemble members can share a bias or interact nonlinearly. The practical goal is a pipeline whose explicit tests track intended use, with residual uncertainty kept visible.
Source: Version-of-record PDF pages 30 to 32, Section 5, Version-of-record PDF pages 44 to 61, Appendices B to G
Source map
These are the source locations that carry the argument. Use them to check this explanation against the original rather than trusting the summary alone.
| Locus | Why it matters | Source |
|---|---|---|
| Version-of-record PDF pages 2 to 4, Section 1 | Defines an ML pipeline, its standard validation criterion, and underspecification as many predictors with equivalent test performance but different deployment behavior. | Open source → |
| Version-of-record PDF pages 4 to 7, Sections 2.1 to 2.3 | Formalizes the predictor set, separates structural failure from underspecification, and motivates application-specific stress tests. | Open source → |
| Version-of-record PDF pages 7 to 11, Section 3 and Figures 1 to 3 | Demonstrates non-identification in epidemic forecasting, genomic transfer, and random-feature models with equivalent standard risk. | Open source → |
| Version-of-record PDF pages 11 to 13, Section 4.1 | States the retraining and stress-test protocol, explains why random seeds probe the equivalence set, and warns that a finite ensemble underestimates it. | Open source → |
| Version-of-record PDF pages 13 to 16, Section 4.2, Figures 4 and 5, and Table 1 | Reports near-identical ImageNet validation accuracy across retrainings and larger variation on corruptions and ObjectNet. | Open source → |
| Version-of-record PDF pages 15 to 26, Sections 4.3 and 4.4, Figures 6 to 10, and Tables 2 to 4 | Tests subgroup calibration and sensitivity to shortcut features in medical imaging and language models across random pretraining and fine-tuning runs. | Open source → |
| Version-of-record PDF pages 27 to 30, Section 4.5, Figures 11 and 12, and Tables 5 and 6 | Examines clinical risk models across sites, years, racial groups, and feature-removal stress tests using electronic health records. | Open source → |
| Version-of-record PDF pages 30 to 32, Section 5 | Interprets random-seed studies as lower bounds, explains why model averaging is not a general solution, and calls for explicit deployment requirements. | Open source → |
| Version-of-record PDF pages 44 to 61, Appendices B to G | Supplies experiment designs, hyperparameters, data definitions, and extended results for the empirical case studies. | Open source → |
The Assumption Switch
One result. One assumption exposed. Turn it and see what changes.
Assumption under test
Standard IID validation constrains every behavior that matters in deployment.
- Held in the source
- If the validation criterion covers the relevant deployment behavior, models with equivalent validation performance are interchangeable for the intended use.
- Turn it
- If validation leaves a deployment-relevant behavior unconstrained, small pipeline perturbations can choose different models that remain tied on the standard test but separate under a stress test.
- What changes
- Model selection then depends on arbitrary pipeline details. A single successful run supplies weak evidence about the behavior of the next model trained by the same procedure.
The common misreading
Random seeds do not cause underspecification. They expose it by selecting different solutions left open by the training and validation pipeline. The paper also separates underspecification from a structural mismatch where every model that performs well on the training distribution must fail a deployment requirement.
Outside the ML frame
Metrology and quality engineering
Does a passing measurement control the property users actually depend on, or only a convenient surrogate under laboratory conditions?
A manufacturing specification can accept many parts that agree on measured dimensions but differ under heat, vibration, or aging. Engineers use tolerance analysis and environmental stress tests to characterize that hidden variation. The paper makes an analogous move for learned predictors: treat the pipeline as a process that returns a distribution of products, then test the requirements that define use. This is our interpretation, not a transfer result established by the paper.
Where the result stops
The empirical studies perturb selected sources of randomness and therefore sample only a small, convenient part of the full equivalence set. Stress tests are application-specific and can miss unanticipated deployment conditions. Variation across retrainings diagnoses underspecification but does not by itself reveal the causal feature or prescribe a repair. The paper does not claim that averaging models, choosing the best standard validation score, or adding any single robustness metric will reliably solve the problem.
The numbers, with their measurands
Each value below states its measurand, evidence type, source location and evidence base when the source reports one. These fields distinguish self reported results from independent measurements.
- 0.001 versus 0.024. standard deviation of accuracy proportions across ResNet-50 retrainings on ImageNet validation versus the pixelation stress test. Reported as measured, Section 4.2, Table 1, version-of-record PDF page 16. Evidence base: 50 independently trained ResNet-50 models. Check it →
- 0.135, 95% CI 0.070 to 0.20. Spearman correlation between British and non-British evaluation performance for polygenic risk scores. Reported as measured, Section 3.2 and Figure 2, version-of-record PDF pages 9 to 10. Evidence base: 1,000 polygenic risk score predictors. Check it →
What remains open
- How should teams choose stress tests when the most important deployment shifts are not known in advance?
- What sampling scheme gives useful coverage of the models a modern training pipeline can return?
- When can additional constraints shrink the equivalence set without sacrificing standard predictive performance?
- How should a safety case represent residual uncertainty after a model passes every currently known stress test?
What this bears on
Superalignment maintains a public register of the claims it makes and the evidence that would change its mind. The entries below connect this source to the exact public claims it bears on.
- C4. Behavioral evaluation cannot carry a deployment decision alone. This record supports it, directly. Across multiple domains, standard validation equivalence leaves deployment behavior unresolved, so a safety claim needs stress evidence that matches its intended operating envelope. See the claim and what would change our mind →
Source audit and review status
| Field | Result | Checked | By | Against |
|---|---|---|---|---|
title | exact | 2026-08-17 | codex-primary-source-review | source |
authors | exact | 2026-08-17 | codex-primary-source-review | source |
date | exact | 2026-08-17 | codex-primary-source-review | source |
venue | exact | 2026-08-17 | codex-primary-source-review | source |
full_text | exact | 2026-08-17 | codex-primary-source-review | source |
- Review status prototype.
- Program collection seminal v1; mechanism study; wave 3, release slot unassigned.
- Source access public full text, PDF. Open the reading copy →
- Provider-authored safety claim yes.
- Explained by Superalignment Research.
- Reviewed by No named human reviewer yet.
- Explainer dates created 2026-08-17; updated 2026-08-17.
- AI assistance AI assisted with canonical-source retrieval, full-text extraction, page rendering, section-by-section reading, locus checking, first-pass prose, figure design, and implementation. The page remains a prototype until a named human review is recorded.
- Rights and access The JMLR version of record is publicly available under CC BY 4.0. This prototype read the complete 61-page publisher PDF and treats arXiv:2011.03395 as an earlier manifestation of the same work.
- Corrections Read the correction policy or report an error.
Provenance
- First seen 2026-08-16, via seed_library.py, arxiv shard of the Stampy snapshot.
- Work id
work:underspecification-presents-challenges-for-credibility-in-modern-machine-learning, which groups manifestations of the same intellectual work. - Record id
arxiv:2011.03395, the natural key for this catalog manifestation. - 2026-08-16 seeded from the arxiv shard
- 2026-08-17 full JMLR version-of-record read and implementation-ready Explained prototype prepared
Full audit data, including this record under id
arxiv:2011.03395:
/library/records.jsonl.
Compact browser index:
/library/corpus.json.
Catalog method and counts:
/library/index.json.