Wiki · Reference entry
Recursive self-improvement
AI systems doing the research that produces better AI systems.
Compound interest starts slow and then speeds up, because each year's gains begin earning gains of their own. Recursive self-improvement is the same loop applied to capability. If AI systems get good at the work of building better AI, each improvement can speed up the next one. Nobody knows whether such a loop would run away, level off, or stall on limits of hardware and data. It is one of the routes people have in mind when they talk about superintelligence arriving quickly.
Why it matters
Oversight depends on having time to notice a problem and respond. A faster loop leaves less of that time between one generation of systems and the next.