Convergence Programming · Chapter 3 · 4 min
The failure gets a name
Chapters
Explore all seven figuresYou cannot engineer against a failure with no name.
A trajectory exhibits false convergence when visible observations, user acceptance, model confidence or passing tests suggest readiness, while the behavior the world actually requires remains unsafe, unsatisfied, unmeasured or untested.
It is not that the artifact is wrong. It is that the process manufactured the appearance of readiness without producing the evidence that would justify it. Worse than an absence of evidence: a green dashboard can be produced either by satisfying requirements or by never discovering them, which means a process can be optimized to benefit from not looking.
Watch a discovery change the known gap
Follow an explicit four-requirement studio example. The chart counts unresolved requirements, rather than inventing a percentage of real-world readiness. Step through discovery and repair, or choose another possible trajectory.
Four requirements define this synthetic studio world. Opening hours and session duration are initially known; a preparation gap and cancellation cutoff are hidden. Discovering an unmet requirement raises the known unresolved count but does not change the behavior. Repair changes satisfaction and lowers the count. The optional evaluator view reveals the full fixed requirement set for illustration; a deployed system does not possess that view. Five selectable trajectories show true convergence, false convergence, plateau, regression and premature action.
A zero known gap means the known requirements are satisfied in the shown scenario. It does not establish that every relevant requirement has been discovered. Surfacing an unmet rule is information progress; repairing and checking the behavior is a separate step. The evaluator can compare both because this small task world was specified in advance.
True convergence may surface bad news. False convergence keeps the dashboard green.
Six ways it happens
- Structural false convergenceA requirement was never elicited, so it was never tested. Nothing went wrong; nothing was asked.
- Proxy false convergenceAn observable metric improves while true behavior does not. The score was never the thing.
- Persuasive false convergenceAcceptance rises because the system became more confident or more agreeable, not more correct.
- Adversarial false convergenceBehavior looks safe under oversight and differs under conditions oversight never creates.
- Regression false convergenceA repair fixes the newest visible issue and silently breaks something established earlier.
- Envelope false convergenceEvidence from one world, policy version, tool set or data distribution is treated as readiness in another.
Five of those six need no bad actor. The adversarial case matters more as systems become more capable, more strategic, or more optimized for appearing safe. The chest-pain patient was structural; the text message was structural too; the repair that would have deleted the visit reason to fix privacy, and broken duplicate matching in the process, would have been regression.
For builders
Separate three numbers on your own dashboards and never let one stand in for another: how the system does on the requirements you know about, how many requirements you have surfaced so far, and what you have done about the ones you have reason to believe exist but have not found. The first can worsen when the process discovers an unmet requirement. Treat that as information gained, then check whether repair resolves it. Do not reward a dashboard for avoiding useful observations.
For researchers
The paper represents the latent operational target with an evaluator ledger: anchors with weights, severities, visibility status and satisfaction scores over trajectory time. Three weighted residual gaps follow: the full gap over all anchors, the known gap over anchors revealed so far, and the unrevealed gap over the rest. A large difference between known and full gaps is a false-convergence signature. The full ledger is an evaluator instrument, available to a benchmark and not to the acting system. Definitions and conventions are on the paper page.