Superalignment

An agent is given a goal, then plans, calls tools, reads results and tries again until it decides it is done. The difference from a chatbot is consequence. A chatbot writes a refund email for you to send. An agent issues the refund. Everything that makes agents valuable comes from that, and so does everything that makes them frightening.

Why it matters

The moment software acts, someone has to be accountable for what it did.