Research | Convergence Programming

The regime we are already in

Software can now be produced from a sentence, by someone who could not read the result, and it can act before anyone has established what it was supposed to do.

That combination is new. Iteration is not new, and neither is testing, requirements discovery or assurance. What changed is that language models produce executable behavior directly from lossy signals. A non-programmer can generate an artifact they cannot inspect. An agent can act through tools whose side effects exceed anything the conversation would reveal.

The consequence is a gap between a weak inspector and a strong generator. The person who knows the work cannot read the implementation. The system that wrote it can act faster than the person can check. Neither party is doing anything wrong, and the result is still unsafe.

The unit of programming stops being the artifact and becomes the trajectory: prompts, procedures, actions, observations, repairs and decisions over time.

False convergence

The failure we named, because you cannot build against a failure with no name.

A trajectory exhibits false convergence when visible observations, user acceptance, model confidence or passing tests suggest readiness, while the behavior the world actually requires remains unsafe, unsatisfied, unmeasured or untested.

It is not that the artifact is wrong. It is that the process manufactured the appearance of readiness without producing the evidence that would justify it. And it is worse than an absence of evidence: a green dashboard can be produced either by satisfying requirements or by never discovering them, which means a process can be optimised to benefit from not looking.

Structural false convergenceA requirement was never elicited, so it was never tested. Nothing went wrong; nothing was asked.
Proxy false convergenceAn observable metric improves while true behavior does not. The score was never the thing.
Persuasive false convergenceAcceptance rises because the system became more confident or more agreeable, not more correct.
Adversarial false convergenceBehavior looks safe under oversight and differs under conditions oversight never creates.
Regression false convergenceA repair fixes the newest visible issue and silently breaks something established earlier.
Envelope false convergenceEvidence from one world, policy version, tool set or data distribution is treated as readiness in another.

Five of those six need no bad actor. The adversarial case matters more as systems become more capable, more strategic, or more optimised for appearing safe.

Watch the dashboard lie

This is the shape of the problem in one figure. Drag the slider from left to right to surface hidden requirements, and watch three quantities that people routinely confuse.

At the left, visible readiness is high and the known-ledger gap is small, because almost nothing has been asked. The full gap is wide and invisible. As discovery proceeds, the visible score gets worse while the system gets genuinely better, because a newly surfaced requirement arrives unmet. Any process that rewards the dashboard will punish the discovery.

True convergence may surface bad news. False convergence keeps the dashboard green.

Five things the same loop can do

One human, one system, one world, iterating. Where it ends up depends entirely on what the trajectory discovers and what it manages to preserve. Only the first of these is a success, and only two of the other four announce themselves while they are happening.

Four structural limits

These are not curiosities. Read as engineering constraints, they say what no amount of model capability will buy you. The formal statements live in the paper; the load-bearing ideas are here.

Specification debt: A prompt is an information bottleneck. Constraints neither encoded in it nor revealed by later interaction are simply absent, and no amount of model capability turns unobserved world structure into evidence. The hard requirements are the idiosyncratic ones: this organization, this policy, this situation.
No free readiness: If two possible worlds produce the same observation history but demand different safety decisions, no rule that sees only that history can certify both. The consequence is not that safety is impossible. It is that the distinguishing observation has to be made, elicited, monitored, or carried openly as risk.
Critical blockers do not average: One unresolved critical blocker changes the type of decision rather than lowering a score. A large enough pile of routine successes will hide a single critical failure under any average-gap threshold, which is why readiness is a checklist and not a mean.
Autonomy spends evidence: Every accepted action draws down a finite budget of residual uncertainty. A system safe enough to draft one email may be unsafe to run a support desk for a month without reopening the loop. Long-lived agency is a sequence of evidence-bounded windows, not one standing approval.

Why more critics stop helping

If autonomy spends a finite budget, how many actions does a given body of evidence actually license? The paper answers with a conditional action window. It is worth playing with, because it makes one thing obvious that intuition gets wrong: verifier multiplicity is not verifier independence.

Shared model families, shared prompts, shared training data and shared evaluative frames impose a floor. Past that floor, adding critics improves the evidence dashboard and changes nothing about the correct decision. The implication is receding-horizon oversight: long-lived agency as a sequence of evidence-bounded windows, never one indefinitely valid approval resting on stale evidence.

A harness shaped by its own theorems

The limits are only negative if the trajectory stays passive. Read constructively they prescribe an architecture, and the prescription is unusually literal: every component exists because a specific limit shows what fails when it is missing. Remove any one and the corresponding failure mode returns.

The minimal version does four things a prompt-to-artifact system does not: it records assumptions as tentative anchors, actively seeks observations that could falsify readiness, turns every discovered anchor into a standing regression obligation, and gates action by the scope of its evidence rather than by model confidence or a user's approval.

Convergence infrastructure does not eliminate uncertainty. It prevents uncertainty from being mistaken for readiness.

Why the obvious remedies are not enough

Each of these is useful. Each is individually insufficient, and for the same underlying reason: none of them certifies a distinction that never entered the observation history.

Remedy	What it leaves open
Prompt engineering	Cannot recover information that never entered the observation history.
Polished demos	Answer usability. They do not distinguish hidden critical scenarios.
Self-reflection	Without independent observation it collapses into correlated self-report.
Model-as-judge	Improves explanations. Shares the blind spots of the thing it judges.
Generated tests	Preserve known anchors, and make false convergence more convincing when generated from the same underspecified prompt.
Retrieval	Adds documents. It does not execute the scenario that would have failed.
Human approval	Decisive but weak where the inspector cannot read the implementation.
Formal verification	Certifies against a specification, which is precisely the thing that is missing.
Runtime monitoring	Catches what has already happened, which for irreversible action is too late.

Bounded on purpose

Convergence Programming is bounded not because it is small, but because bounded worlds make oversight measurable.

This is the epistemic contract, and we would rather state it plainly than have it discovered. Every boundary below is a place where the framework declines to claim something.

Boundary	What is assumed	Why it matters
Task	A scoped executable world, not the whole of human value.	Prevents overclaiming.
World	An environment that can be modelled, simulated, sampled or operationally scoped.	Makes grounding possible at all.
Ledger	A hidden but finite evaluator target.	Makes scoring possible.
Observation	Claims limited by what the history actually contains.	Gives the non-certification result its force.
Action	A defined action surface.	Lets risk scale with autonomy.
Budget	Oversight costs questions, scenarios, experts, tools, compute.	Connects to scalable supervision.
Envelope	Evidence valid only under stated conditions.	Handles drift and deployment shift.

What this is not: a theory of all human values, a replacement for formal methods, a claim that natural language is a sufficient specification, or an assumption that every hidden constraint is discoverable and every repair loop converges. The boundedness is not a retreat from the alignment problem. It is what turns abstract oversight failures into measurable trajectory failures you can run today.

What we are building toward

The target is not maximal uninterrupted autonomy. It is calibrated permission to act: discover the hidden constraints, repair and preserve them, bind every readiness claim to independent evidence, act inside the envelope that evidence covers, and reopen grounding when the budget runs out.

Reliable AI will require better generation. It will require something deeper as well: better convergence.

Request the paper Why a three-body problem

The paper is a conceptual and evaluation-design contribution with a preliminary pilot. It is not a benchmark or a leaderboard, and the figures on this page are illustrations of its arguments rather than measured results.