Wiki · Reference entry
Observability
Being able to see what the system actually did, after the fact.
For a system that reasons, logs are not enough. You want the inputs, the steps, the tools it called, the intermediate conclusions and the evidence behind the final answer, kept in a form that answers both the engineer's post-mortem and the auditor's question, which are not the same question.
Why it matters
If you cannot reconstruct a decision, you cannot defend it.