Superalignment

Autonomy is a dial, not a switch, and the honest way to set it is by what has been proven rather than by how confident the system sounds. A system that has been shown to handle a case correctly can be allowed to handle that case. It has not earned the next case, and the two are easy to confuse.

Why it matters

Most incidents are an autonomy setting nobody made deliberately.